[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:16 dist-test-slave-gjw7 krb5kdc[3779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:16 dist-test-slave-gjw7 krb5kdc[3779](info): set up 2 sockets
Aug 08 04:58:16 dist-test-slave-gjw7 krb5kdc[3779](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 04:58:17.693128 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:17 dist-test-slave-gjw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:17.818539 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:42413
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:42413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:17.909185 3807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:17.909432 3807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:17.909458 3807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:17.910565 3807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:17.910598 3807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:17.910611 3807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:17.910624 3807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:17.910633 3807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:17.912174 3807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:42413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:42413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:17.912492 3807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:17.912755 3807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:17.918430 3817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:17.918494 3807 server_base.cc:1034] running on GCE node
W20260808 04:58:17.919427 3820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:17.919512 3815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:17.919863 3807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:17.920214 3807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:17.921367 3807 hybrid_clock.cc:648] HybridClock initialized: now 1786165097921327 us; error 63 us; skew 500 ppm
Aug 08 04:58:17 dist-test-slave-gjw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165097, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:17.924068 3807 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:17.924664 3807 webserver.cc:460] Webserver started at http://127.3.174.190:35775/ using document root <none> and password file <none>
I20260808 04:58:17.924891 3807 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:17.924942 3807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:17.925071 3807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:17.926407 3807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9"
format_stamp: "Formatted at 2026-08-08 04:58:17 on dist-test-slave-gjw7"
server_key: "0c474f2db6fe71e7a0f12fc39929ab69"
server_key_iv: "3cfee2cfa805ff5c1d66cd237baf6886"
server_key_version: "encryptionkey@0"
I20260808 04:58:17.927093 3807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9"
format_stamp: "Formatted at 2026-08-08 04:58:17 on dist-test-slave-gjw7"
server_key: "0c474f2db6fe71e7a0f12fc39929ab69"
server_key_iv: "3cfee2cfa805ff5c1d66cd237baf6886"
server_key_version: "encryptionkey@0"
I20260808 04:58:17.928771 3807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:17.929665 3828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:17.929824 3807 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:17.929919 3807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9"
format_stamp: "Formatted at 2026-08-08 04:58:17 on dist-test-slave-gjw7"
server_key: "0c474f2db6fe71e7a0f12fc39929ab69"
server_key_iv: "3cfee2cfa805ff5c1d66cd237baf6886"
server_key_version: "encryptionkey@0"
I20260808 04:58:17.930027 3807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:17.965130 3807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:17.967891 3807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:17.968127 3807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:17.973838 3807 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:42413
I20260808 04:58:17.973891 3930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:42413 every 8 connection(s)
I20260808 04:58:17.974355 3807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:17.975610 3933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:17.978400 3933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9: Bootstrap starting.
I20260808 04:58:17.979199 3933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:17.979554 3933 log.cc:824] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:17.980315 3933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9: No bootstrap required, opened a new log
I20260808 04:58:17.983289 3933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42413 } }
I20260808 04:58:17.983443 3933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:17.983522 3933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8e605f5c50407d8e75dbf3d8a66ab9, State: Initialized, Role: FOLLOWER
I20260808 04:58:17.983760 3933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42413 } }
I20260808 04:58:17.983881 3933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:17.983913 3933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:17.983971 3933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:17.984108 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 3807
I20260808 04:58:17.984217 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:17.984594 3933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42413 } }
I20260808 04:58:17.984706 3933 leader_election.cc:302] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e8e605f5c50407d8e75dbf3d8a66ab9; no voters:
I20260808 04:58:17.984898 3933 leader_election.cc:288] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:17.984943 3938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:17.985088 3770 external_mini_cluster.cc:1428] Setting key 266d65079cd45bcd8adb05e9b3038143
I20260808 04:58:17.985168 3938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [term 1 LEADER]: Becoming Leader. State: Replica: 3e8e605f5c50407d8e75dbf3d8a66ab9, State: Running, Role: LEADER
I20260808 04:58:17.985342 3933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:17.985267 3938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42413 } }
I20260808 04:58:17.986631 3941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e8e605f5c50407d8e75dbf3d8a66ab9. Latest consensus state: current_term: 1 leader_uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42413 } } }
I20260808 04:58:17.986743 3941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:17.987905 3940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8e605f5c50407d8e75dbf3d8a66ab9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42413 } } }
I20260808 04:58:17.987987 3940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:17.988058 3953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:17.988946 3953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:17.991796 3953 catalog_manager.cc:1347] Generated new cluster ID: 0eb3ee56f6684f44be3cf50d9d43a8f3
I20260808 04:58:17.991844 3953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 04:58:17 dist-test-slave-gjw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:18.004081 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:17.986378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.3.174.190:42413)
0808 04:58:17.987617 (+ 1239us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:17.987626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:17.989360 (+ 1734us) server_negotiation.cc:407] Connection header received
0808 04:58:17.989927 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:17.989934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:17.989994 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:17.990144 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:17.991799 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:17.993320 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:17.994839 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:17.995434 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:17.995517 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:18.000110 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:18.000114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:18.000115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:18.000128 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:18.002174 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.002631 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.002635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.002635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.002752 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.002983 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.002984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.002984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.003287 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:18.003444 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:18.003694 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 04:58:18.003837 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":995,"thread_start_us":349,"threads_started":1}
I20260808 04:58:18.008525 3953 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:18.009037 3953 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:18.012600 3953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e8e605f5c50407d8e75dbf3d8a66ab9: Generated new TSK 0
I20260808 04:58:18.012770 3953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:18.040619 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42413
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:18.147562 3977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:18.147784 3977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:18.147820 3977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:18.149014 3977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:18.149056 3977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:18.149091 3977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:18.150705 3977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42413
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:18.151016 3977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:18.151242 3977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:18.157356 3987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:18.157336 3986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:18.157326 3990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:18.157907 3977 server_base.cc:1034] running on GCE node
I20260808 04:58:18.158097 3977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:18.158347 3977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:18.159504 3977 hybrid_clock.cc:648] HybridClock initialized: now 1786165098159488 us; error 35 us; skew 500 ppm
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:18.162674 3977 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:18.163115 3977 webserver.cc:460] Webserver started at http://127.3.174.129:42361/ using document root <none> and password file <none>
I20260808 04:58:18.163295 3977 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:18.163333 3977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:18.163425 3977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:18.164618 3977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "48a52b19a05b4c379cc113cbc94c8a9a"
format_stamp: "Formatted at 2026-08-08 04:58:18 on dist-test-slave-gjw7"
server_key: "8523c21183e05b73b365265eb475a48c"
server_key_iv: "dcfefb2827e098f8278499dcdaef0b92"
server_key_version: "encryptionkey@0"
I20260808 04:58:18.164974 3977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "48a52b19a05b4c379cc113cbc94c8a9a"
format_stamp: "Formatted at 2026-08-08 04:58:18 on dist-test-slave-gjw7"
server_key: "8523c21183e05b73b365265eb475a48c"
server_key_iv: "dcfefb2827e098f8278499dcdaef0b92"
server_key_version: "encryptionkey@0"
I20260808 04:58:18.166523 3977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:18.167450 4000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:18.167627 3977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:18.167714 3977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "48a52b19a05b4c379cc113cbc94c8a9a"
format_stamp: "Formatted at 2026-08-08 04:58:18 on dist-test-slave-gjw7"
server_key: "8523c21183e05b73b365265eb475a48c"
server_key_iv: "dcfefb2827e098f8278499dcdaef0b92"
server_key_version: "encryptionkey@0"
I20260808 04:58:18.167776 3977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:18.192873 3977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:18.195035 3977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:18.195154 3977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:18.195456 3977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:18.195884 3977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:18.195968 3977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:18.195997 3977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:18.196012 3977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:18.211128 3977 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:38767
I20260808 04:58:18.211272 4214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:38767 every 8 connection(s)
I20260808 04:58:18.211544 3977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:58:18.217236 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 3977
I20260808 04:58:18.217355 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:18.217511 3770 external_mini_cluster.cc:1428] Setting key af09e83ba9ca7159994f0c749e5f8ea6
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:18.222084 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:18.212562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:56911 (local address 127.3.174.190:42413)
0808 04:58:18.212609 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:18.212611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:18.212941 (+ 330us) server_negotiation.cc:407] Connection header received
0808 04:58:18.213548 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:18.213549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:18.213560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:18.213613 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:18.214875 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.215378 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.216565 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.216773 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.216823 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:18.219423 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:18.219425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:18.219425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:18.219433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:18.220747 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.221119 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.221120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.221121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.221176 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.221438 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.221439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.221439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.221552 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:18.221609 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:18.221882 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 04:58:18.221988 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 04:58:18.222234 4217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:18.212530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:42413 (local address 127.3.174.129:56911)
0808 04:58:18.212795 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:18.212834 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:18.213456 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:18.213671 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:18.213675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:18.214481 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:18.214797 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:18.214804 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.215484 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.215486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:18.216483 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:18.216486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.216839 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.216840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:18.216954 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:18.217634 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:18.217645 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:18.219332 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:18.220821 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:18.220828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:18.220838 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:18.221050 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:18.221232 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:18.221233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:18.221233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:18.221381 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:18.221608 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:18.221615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:18.221750 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 04:58:18.221833 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 04:58:18.221989 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":153,"threads_started":1}
I20260808 04:58:18.222791 4215 heartbeater.cc:347] Connected to a master server at 127.3.174.190:42413
I20260808 04:58:18.222883 4215 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:18.223062 4215 heartbeater.cc:511] Master 127.3.174.190:42413 requested a full tablet report, sending...
I20260808 04:58:18.223690 3854 ts_manager.cc:194] Registered new tserver with Master: 48a52b19a05b4c379cc113cbc94c8a9a (127.3.174.129:38767)
I20260808 04:58:18.224363 3854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:56911
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:18.247768 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42413
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:18.357765 4222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:18.357954 4222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:18.357976 4222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:18.359131 4222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:18.359169 4222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:18.359201 4222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:18.360860 4222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42413
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:18.361100 4222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:18.361367 4222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:18.369819 4228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:18.369889 4233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:18.370040 4230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:18.370275 4222 server_base.cc:1034] running on GCE node
I20260808 04:58:18.370476 4222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:18.370735 4222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:18.371882 4222 hybrid_clock.cc:648] HybridClock initialized: now 1786165098371868 us; error 26 us; skew 500 ppm
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:18.374884 4222 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:18.375469 4222 webserver.cc:460] Webserver started at http://127.3.174.130:37439/ using document root <none> and password file <none>
I20260808 04:58:18.375696 4222 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:18.375743 4222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:18.375864 4222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:18.377027 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "f37ff4f000b2454a82c8b0de995ba12e"
format_stamp: "Formatted at 2026-08-08 04:58:18 on dist-test-slave-gjw7"
server_key: "2c7ead5e3dc1e3e17c8fb42840d2e40b"
server_key_iv: "b7f57d57d79ec802929ecc77f167de67"
server_key_version: "encryptionkey@0"
I20260808 04:58:18.377416 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "f37ff4f000b2454a82c8b0de995ba12e"
format_stamp: "Formatted at 2026-08-08 04:58:18 on dist-test-slave-gjw7"
server_key: "2c7ead5e3dc1e3e17c8fb42840d2e40b"
server_key_iv: "b7f57d57d79ec802929ecc77f167de67"
server_key_version: "encryptionkey@0"
I20260808 04:58:18.378918 4222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:18.379750 4243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:18.379902 4222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:18.379987 4222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "f37ff4f000b2454a82c8b0de995ba12e"
format_stamp: "Formatted at 2026-08-08 04:58:18 on dist-test-slave-gjw7"
server_key: "2c7ead5e3dc1e3e17c8fb42840d2e40b"
server_key_iv: "b7f57d57d79ec802929ecc77f167de67"
server_key_version: "encryptionkey@0"
I20260808 04:58:18.380054 4222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:18.389967 4222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:18.391894 4222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:18.392077 4222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:18.392491 4222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:18.392843 4222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:18.392879 4222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:18.392906 4222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:18.392925 4222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:18.408927 4222 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:40907
I20260808 04:58:18.408998 4445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:40907 every 8 connection(s)
I20260808 04:58:18.409395 4222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:58:18.415679 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 4222
I20260808 04:58:18.415768 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:18.415941 3770 external_mini_cluster.cc:1428] Setting key 0654877417ebc9cb56a59e026af8ce21
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:18.419896 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:18.410258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:35749 (local address 127.3.174.190:42413)
0808 04:58:18.410341 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:18.410345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:18.410770 (+ 425us) server_negotiation.cc:407] Connection header received
0808 04:58:18.411593 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:18.411594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:18.411605 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:18.411655 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:18.412813 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.413368 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.414505 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.414694 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.414746 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:18.417236 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:18.417237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:18.417237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:18.417245 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:18.418574 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.418956 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.418957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.418957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.419017 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.419269 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.419270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.419271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.419389 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:18.419445 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:18.419710 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 04:58:18.419798 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 04:58:18.420084 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:18.410241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:42413 (local address 127.3.174.130:35749)
0808 04:58:18.410596 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:18.410665 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:18.411502 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:18.411720 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:18.411724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:18.412370 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:18.412729 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:18.412735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.413440 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.413441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:18.414437 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:18.414440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.414761 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.414762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:18.414904 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:18.415433 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:18.415442 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:18.417089 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:18.418652 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:18.418659 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:18.418661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:18.418887 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:18.419070 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:18.419071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:18.419071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:18.419206 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:18.419445 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:18.419452 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:18.419590 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 04:58:18.419688 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 04:58:18.419850 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":212,"threads_started":1}
I20260808 04:58:18.420568 4449 heartbeater.cc:347] Connected to a master server at 127.3.174.190:42413
I20260808 04:58:18.420682 4449 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:18.420866 4449 heartbeater.cc:511] Master 127.3.174.190:42413 requested a full tablet report, sending...
I20260808 04:58:18.421262 3854 ts_manager.cc:194] Registered new tserver with Master: f37ff4f000b2454a82c8b0de995ba12e (127.3.174.130:40907)
I20260808 04:58:18.421648 3854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:35749
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:18.443939 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42413
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:18.562614 4456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:18.562815 4456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:18.562849 4456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:18.564008 4456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:18.564049 4456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:18.564083 4456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:18.565805 4456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42413
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:18.566056 4456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:18.566275 4456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:18.571789 4464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:18.571827 4469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:18.571965 4463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:18.572434 4456 server_base.cc:1034] running on GCE node
I20260808 04:58:18.572633 4456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:18.572937 4456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:18.574066 4456 hybrid_clock.cc:648] HybridClock initialized: now 1786165098574055 us; error 30 us; skew 500 ppm
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:18.577186 4456 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:18.577682 4456 webserver.cc:460] Webserver started at http://127.3.174.131:34627/ using document root <none> and password file <none>
I20260808 04:58:18.577903 4456 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:18.577947 4456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:18.578076 4456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:18.579402 4456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "47d79d4715384ea882d8f70946ef0d4e"
format_stamp: "Formatted at 2026-08-08 04:58:18 on dist-test-slave-gjw7"
server_key: "6cab27bac138b62dbec6241beece9cfc"
server_key_iv: "0a70b34b12ea073c94b1378bb265cae1"
server_key_version: "encryptionkey@0"
I20260808 04:58:18.579800 4456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "47d79d4715384ea882d8f70946ef0d4e"
format_stamp: "Formatted at 2026-08-08 04:58:18 on dist-test-slave-gjw7"
server_key: "6cab27bac138b62dbec6241beece9cfc"
server_key_iv: "0a70b34b12ea073c94b1378bb265cae1"
server_key_version: "encryptionkey@0"
I20260808 04:58:18.581218 4456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:18.581981 4483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:18.582136 4456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:18.582219 4456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "47d79d4715384ea882d8f70946ef0d4e"
format_stamp: "Formatted at 2026-08-08 04:58:18 on dist-test-slave-gjw7"
server_key: "6cab27bac138b62dbec6241beece9cfc"
server_key_iv: "0a70b34b12ea073c94b1378bb265cae1"
server_key_version: "encryptionkey@0"
I20260808 04:58:18.582286 4456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:18.589916 4456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:18.591938 4456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:18.592067 4456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:18.592361 4456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:18.592700 4456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:18.592734 4456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:18.592754 4456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:18.592784 4456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:18.609638 4456 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:36973
I20260808 04:58:18.609686 4684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:36973 every 8 connection(s)
I20260808 04:58:18.610035 4456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:18.619850 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 4456
I20260808 04:58:18.619889 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:18.610798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:50407 (local address 127.3.174.190:42413)
0808 04:58:18.610846 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:18.610848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:18.611179 (+ 331us) server_negotiation.cc:407] Connection header received
0808 04:58:18.611977 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:18.611978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:18.611987 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:18.612049 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:18.613303 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.613813 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.614909 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.615090 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.615138 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:18.617488 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:18.617490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:18.617491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:18.617498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:18.618739 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.619045 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.619046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.619046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.619093 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.619277 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.619278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.619278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.619378 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:18.619426 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:18.619759 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 04:58:18.619823 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:58:18.619941 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:18.620091 3770 external_mini_cluster.cc:1428] Setting key 46810d90eb129c0794ec0e31c4e4b6d6
I20260808 04:58:18.620121 4687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:18.610770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:42413 (local address 127.3.174.131:50407)
0808 04:58:18.610986 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:18.611030 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:18.611890 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:18.612110 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:18.612114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:18.612839 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:18.613160 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:18.613165 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.613884 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.613885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:18.614837 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:18.614840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.615145 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.615146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:18.615325 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:18.615859 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:18.615868 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:18.617396 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:18.618802 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:18.618805 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:18.618807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:18.618995 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:18.619135 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:18.619135 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:18.619135 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:18.619219 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:18.619460 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:18.619466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:18.619600 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 04:58:18.619719 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 04:58:18.619888 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260808 04:58:18.620725 4685 heartbeater.cc:347] Connected to a master server at 127.3.174.190:42413
I20260808 04:58:18.620836 4685 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:18.621012 4685 heartbeater.cc:511] Master 127.3.174.190:42413 requested a full tablet report, sending...
I20260808 04:58:18.621400 3854 ts_manager.cc:194] Registered new tserver with Master: 47d79d4715384ea882d8f70946ef0d4e (127.3.174.131:36973)
I20260808 04:58:18.621766 3854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:50407
I20260808 04:58:18.632124 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:18.657549 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:18.649816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57554 (local address 127.3.174.129:38767)
0808 04:58:18.650118 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:18.650123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:18.650134 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:58:18.650159 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:18.650161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:18.650176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:18.650246 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:18.650780 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.651498 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.652260 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.652577 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.652634 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:18.654856 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:18.654864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:18.654864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:18.654871 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:18.656308 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.656729 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.656733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.656733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.656805 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.657000 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.657001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.657002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.657146 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:18.657328 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:18.657372 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 04:58:18.657438 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":208,"threads_started":1}
W20260808 04:58:18.658030 4048 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57554
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:18.681097 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:18.674309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.3.174.129:38767)
0808 04:58:18.674369 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:18.674371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:18.674444 (+ 73us) server_negotiation.cc:407] Connection header received
0808 04:58:18.674502 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:18.674502 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:18.674516 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:18.674559 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:18.675180 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.675674 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.676514 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:18.676717 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:18.676770 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:18.679022 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:18.679023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:18.679023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:18.679029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:18.680152 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.680478 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.680479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.680479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.680543 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:18.680692 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:18.680692 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:18.680692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:18.680785 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:18.680877 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:18.680927 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 04:58:18.680995 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 04:58:18.681731 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 3977
I20260808 04:58:18.703039 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 4222
I20260808 04:58:18.716477 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 4456
I20260808 04:58:18.734673 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 3807
2026-08-08T04:58:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2262 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[4706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[4706](info): set up 2 sockets
Aug 08 04:58:18 dist-test-slave-gjw7 krb5kdc[4706](info): commencing operation
krb5kdc: starting...
W20260808 04:58:19.906167 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:19 dist-test-slave-gjw7 krb5kdc[4706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:20.023993 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45117
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:45117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:20.106616 4732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:20.106829 4732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:20.106854 4732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:20.107967 4732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:20.108006 4732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:20.108019 4732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:20.108031 4732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:20.108042 4732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:20.109555 4732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:45117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:20.109784 4732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:20.109992 4732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:20.114457 4745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:20.114673 4732 server_base.cc:1034] running on GCE node
W20260808 04:58:20.116122 4743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:20.116303 4741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:20.116567 4732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:20.116812 4732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:20.117965 4732 hybrid_clock.cc:648] HybridClock initialized: now 1786165100117949 us; error 28 us; skew 500 ppm
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:20.120977 4732 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:20.121520 4732 webserver.cc:460] Webserver started at http://127.3.174.190:35965/ using document root <none> and password file <none>
I20260808 04:58:20.121747 4732 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:20.121796 4732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:20.121918 4732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:20.122911 4732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "bb240caae801c60cb3ed1c2e99ad1205"
server_key_iv: "701beb546934d14da1acc16772e10b5d"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.123275 4732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "bb240caae801c60cb3ed1c2e99ad1205"
server_key_iv: "701beb546934d14da1acc16772e10b5d"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.124492 4732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:20.125231 4760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.125423 4732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:20.125509 4732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "bb240caae801c60cb3ed1c2e99ad1205"
server_key_iv: "701beb546934d14da1acc16772e10b5d"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.125579 4732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:20.144850 4732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:20.146834 4732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:20.146958 4732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:20.150703 4732 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:45117
I20260808 04:58:20.150750 4854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:45117 every 8 connection(s)
I20260808 04:58:20.151064 4732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:20.151927 4857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:20.154330 4857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b: Bootstrap starting.
I20260808 04:58:20.154881 4857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:20.155157 4857 log.cc:824] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:20.155936 4857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b: No bootstrap required, opened a new log
I20260808 04:58:20.157581 4857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45117 } }
I20260808 04:58:20.157719 4857 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:20.157742 4857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96d06b8fb8f141bba03ce9f50c0dbb5b, State: Initialized, Role: FOLLOWER
I20260808 04:58:20.157883 4857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45117 } }
I20260808 04:58:20.157948 4857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:20.157970 4857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:20.158018 4857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:20.158504 4857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45117 } }
I20260808 04:58:20.158595 4857 leader_election.cc:302] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96d06b8fb8f141bba03ce9f50c0dbb5b; no voters:
I20260808 04:58:20.158748 4857 leader_election.cc:288] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:20.158793 4865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:20.158905 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 4732
I20260808 04:58:20.158964 4865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [term 1 LEADER]: Becoming Leader. State: Replica: 96d06b8fb8f141bba03ce9f50c0dbb5b, State: Running, Role: LEADER
I20260808 04:58:20.158985 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:20.159015 4857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:20.159054 4865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45117 } }
I20260808 04:58:20.159143 3770 external_mini_cluster.cc:1428] Setting key 910e2680c22bec2699c73604b387382f
I20260808 04:58:20.159358 4865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96d06b8fb8f141bba03ce9f50c0dbb5b. Latest consensus state: current_term: 1 leader_uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45117 } } }
I20260808 04:58:20.159514 4865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:20.159801 4880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:20.159890 4867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d06b8fb8f141bba03ce9f50c0dbb5b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45117 } } }
I20260808 04:58:20.160075 4867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:20.160398 4880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:20.162384 4880 catalog_manager.cc:1347] Generated new cluster ID: 284f328737e3470ba4cf388bfe217c32
I20260808 04:58:20.162436 4880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:20.172185 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.159866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.3.174.190:45117)
0808 04:58:20.161786 (+ 1920us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.161796 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.161829 (+ 33us) server_negotiation.cc:407] Connection header received
0808 04:58:20.161962 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.161968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.162014 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.162142 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:20.162844 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.164249 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.165247 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.165881 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.165988 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.168622 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:20.168626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:20.168627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:20.168639 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:20.170496 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.170901 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.170904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.170904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.171010 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.171203 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.171204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.171204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.171484 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:20.171628 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.171719 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.171951 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1878,"thread_start_us":70,"threads_started":1}
I20260808 04:58:20.179483 4880 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:20.179961 4880 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:20.182580 4880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96d06b8fb8f141bba03ce9f50c0dbb5b: Generated new TSK 0
I20260808 04:58:20.182716 4880 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:20.211347 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45117
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:20.294878 4907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:20.295071 4907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:20.295094 4907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:20.296257 4907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:20.296298 4907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:20.296324 4907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:20.297806 4907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:20.298033 4907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:20.298255 4907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:20.302990 4929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:20.303007 4934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:20.303103 4907 server_base.cc:1034] running on GCE node
W20260808 04:58:20.302999 4931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:20.303485 4907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:20.303704 4907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:20.304853 4907 hybrid_clock.cc:648] HybridClock initialized: now 1786165100304838 us; error 33 us; skew 500 ppm
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:20.307435 4907 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:20.307816 4907 webserver.cc:460] Webserver started at http://127.3.174.129:33481/ using document root <none> and password file <none>
I20260808 04:58:20.308010 4907 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:20.308050 4907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:20.308144 4907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:20.309108 4907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "f6dc0f05a5884966ac848b79dc6d12c0"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "84ed9861f99a809b597d5df1745718df"
server_key_iv: "d9c7a7e21f2655a8e2d794d409db779f"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.309512 4907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "f6dc0f05a5884966ac848b79dc6d12c0"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "84ed9861f99a809b597d5df1745718df"
server_key_iv: "d9c7a7e21f2655a8e2d794d409db779f"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.311131 4907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:20.311829 4944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.311980 4907 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:20.312062 4907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "f6dc0f05a5884966ac848b79dc6d12c0"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "84ed9861f99a809b597d5df1745718df"
server_key_iv: "d9c7a7e21f2655a8e2d794d409db779f"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.312134 4907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:20.319160 4907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:20.321165 4907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:20.321344 4907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:20.321544 4907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:20.321851 4907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:20.321884 4907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.321905 4907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:20.321928 4907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.327571 4907 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:42347
I20260808 04:58:20.327637 5125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:42347 every 8 connection(s)
I20260808 04:58:20.327900 4907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:20.336493 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 4907
I20260808 04:58:20.336570 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:20.336740 3770 external_mini_cluster.cc:1428] Setting key aec7b24bd3b0aab1735777db5e7d32f5
I20260808 04:58:20.336920 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.328586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:44019 (local address 127.3.174.190:45117)
0808 04:58:20.328628 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.328630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.328835 (+ 205us) server_negotiation.cc:407] Connection header received
0808 04:58:20.330176 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.330178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.330203 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.330253 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:20.330844 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.331345 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.332298 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.332494 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.332548 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.334481 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:20.334483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:20.334483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:20.334491 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:20.335607 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.335986 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.335988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.335988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.336044 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.336250 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.336251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.336252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.336378 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:20.336448 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.336738 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.336836 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 04:58:20.337270 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.328572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45117 (local address 127.3.174.129:44019)
0808 04:58:20.328723 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:20.328758 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:20.330067 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:20.330321 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:20.330325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:20.330564 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:20.330755 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.330760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.331423 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.331424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.332217 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.332219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.332560 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.332561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.332676 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.333076 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:20.333084 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:20.334398 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:20.335681 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:20.335687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:20.335706 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:20.335919 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:20.336097 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:20.336098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:20.336098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:20.336195 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:20.336454 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:20.336461 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:20.336600 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 04:58:20.336697 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 04:58:20.336982 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260808 04:58:20.337821 5126 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45117
I20260808 04:58:20.337915 5126 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:20.338099 5126 heartbeater.cc:511] Master 127.3.174.190:45117 requested a full tablet report, sending...
I20260808 04:58:20.338653 4786 ts_manager.cc:194] Registered new tserver with Master: f6dc0f05a5884966ac848b79dc6d12c0 (127.3.174.129:42347)
I20260808 04:58:20.339221 4786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:44019
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:20.361913 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45117
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:20.445093 5136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:20.445313 5136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:20.445348 5136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:20.446475 5136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:20.446513 5136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:20.446547 5136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:20.448009 5136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:20.448249 5136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:20.448469 5136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:20.453150 5152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:20.453254 5136 server_base.cc:1034] running on GCE node
W20260808 04:58:20.453172 5147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:20.453223 5150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:20.453634 5136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:20.453835 5136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:20.454975 5136 hybrid_clock.cc:648] HybridClock initialized: now 1786165100454957 us; error 36 us; skew 500 ppm
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:20.459721 5136 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:20.460165 5136 webserver.cc:460] Webserver started at http://127.3.174.130:41463/ using document root <none> and password file <none>
I20260808 04:58:20.460397 5136 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:20.460457 5136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:20.460582 5136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:20.461717 5136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "81d644f6c30b4e5fae88157fba6f4acb"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "a8eedaa4b4f9ff467cf8fe9eee0ecf09"
server_key_iv: "5a6d8c2b9fac9ebfb49fdc7e63956232"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.462036 5136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "81d644f6c30b4e5fae88157fba6f4acb"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "a8eedaa4b4f9ff467cf8fe9eee0ecf09"
server_key_iv: "5a6d8c2b9fac9ebfb49fdc7e63956232"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.463268 5136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:20.463950 5163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.464105 5136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:20.464190 5136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "81d644f6c30b4e5fae88157fba6f4acb"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "a8eedaa4b4f9ff467cf8fe9eee0ecf09"
server_key_iv: "5a6d8c2b9fac9ebfb49fdc7e63956232"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.464247 5136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:20.472013 5136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:20.474035 5136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:20.474139 5136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:20.474313 5136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:20.474565 5136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:20.474596 5136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.474614 5136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:20.474632 5136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.480216 5136 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:43615
I20260808 04:58:20.480261 5353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:43615 every 8 connection(s)
I20260808 04:58:20.480558 5136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:20.486708 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 5136
I20260808 04:58:20.486824 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:20.487066 3770 external_mini_cluster.cc:1428] Setting key 82c4f08e9ed3d56c56d2d4b4c424e523
I20260808 04:58:20.489635 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.481229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:44267 (local address 127.3.174.190:45117)
0808 04:58:20.481312 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.481315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.481522 (+ 207us) server_negotiation.cc:407] Connection header received
0808 04:58:20.482421 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.482422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.482432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.482481 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:20.483054 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.483554 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.484474 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.484648 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.484691 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.486725 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:20.486728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:20.486728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:20.486736 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:20.488239 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.488624 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.488625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.488625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.488688 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.488897 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.488897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.488898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.489019 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:20.489082 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.489449 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.489539 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 04:58:20.489735 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.481202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45117 (local address 127.3.174.130:44267)
0808 04:58:20.481408 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:20.481444 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:20.482336 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:20.482539 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:20.482543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:20.482782 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:20.482975 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.482979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.483620 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.483620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.484411 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.484413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.484711 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.484712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.484819 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.485275 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:20.485285 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:20.486628 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:20.488335 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:20.488341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:20.488343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:20.488560 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:20.488746 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:20.488747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:20.488747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:20.488842 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:20.489113 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:20.489120 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:20.489276 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 04:58:20.489433 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 04:58:20.489559 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":126,"threads_started":1}
I20260808 04:58:20.490195 5354 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45117
I20260808 04:58:20.490285 5354 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:20.490454 5354 heartbeater.cc:511] Master 127.3.174.190:45117 requested a full tablet report, sending...
I20260808 04:58:20.490808 4786 ts_manager.cc:194] Registered new tserver with Master: 81d644f6c30b4e5fae88157fba6f4acb (127.3.174.130:43615)
I20260808 04:58:20.491255 4786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:44267
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:20.511978 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45117
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:20.593258 5362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:20.593480 5362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:20.593502 5362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:20.594918 5362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:20.594971 5362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:20.595007 5362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:20.596508 5362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:20.596727 5362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:20.596940 5362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:20.601704 5370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:20.601763 5380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:20.601764 5374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:20.601871 5362 server_base.cc:1034] running on GCE node
I20260808 04:58:20.602124 5362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:20.602342 5362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:20.603477 5362 hybrid_clock.cc:648] HybridClock initialized: now 1786165100603463 us; error 29 us; skew 500 ppm
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:20.606068 5362 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:20.606557 5362 webserver.cc:460] Webserver started at http://127.3.174.131:42323/ using document root <none> and password file <none>
I20260808 04:58:20.606801 5362 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:20.606855 5362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:20.606995 5362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:20.608108 5362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "6d3fc136865448778290a73095b0efd6"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "b6e33f1edfc4240df24f3acb539f7e2d"
server_key_iv: "9deee20781fa0be244768883285d426c"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.608479 5362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "6d3fc136865448778290a73095b0efd6"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "b6e33f1edfc4240df24f3acb539f7e2d"
server_key_iv: "9deee20781fa0be244768883285d426c"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.609885 5362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 04:58:20.610621 5389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.610790 5362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.610872 5362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "6d3fc136865448778290a73095b0efd6"
format_stamp: "Formatted at 2026-08-08 04:58:20 on dist-test-slave-gjw7"
server_key: "b6e33f1edfc4240df24f3acb539f7e2d"
server_key_iv: "9deee20781fa0be244768883285d426c"
server_key_version: "encryptionkey@0"
I20260808 04:58:20.610940 5362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:20.623433 5362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:20.625360 5362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:20.625507 5362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:20.625735 5362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:20.626041 5362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:20.626080 5362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.626112 5362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:20.626140 5362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:20.632673 5362 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:45911
I20260808 04:58:20.632706 5574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:45911 every 8 connection(s)
I20260808 04:58:20.633028 5362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 04:58:20.636619 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 5362
I20260808 04:58:20.636720 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:20.636878 3770 external_mini_cluster.cc:1428] Setting key 9cc91534f5ee0e27d86510e179b55407
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:20.642035 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.633841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:45231 (local address 127.3.174.190:45117)
0808 04:58:20.633904 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.633906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.634077 (+ 171us) server_negotiation.cc:407] Connection header received
0808 04:58:20.635030 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.635039 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.635052 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.635102 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:20.635712 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.636232 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.637352 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.637544 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.637598 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.639561 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:20.639563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:20.639563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:20.639571 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:20.640780 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.641132 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.641133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.641134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.641186 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.641438 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.641439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.641439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.641546 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:20.641601 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.641865 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.641946 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 04:58:20.642179 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.633795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45117 (local address 127.3.174.131:45231)
0808 04:58:20.633968 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:20.634005 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:20.634925 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:20.635176 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:20.635179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:20.635433 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:20.635622 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.635626 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.636316 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.636317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.637248 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.637251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.637610 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.637611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.637743 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.638171 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:20.638179 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:20.639479 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:20.640851 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:20.640857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:20.640859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:20.641068 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:20.641240 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:20.641240 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:20.641241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:20.641380 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:20.641601 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:20.641607 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:20.641744 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 04:58:20.641831 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 04:58:20.641977 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260808 04:58:20.642619 5575 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45117
I20260808 04:58:20.642735 5575 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:20.642907 5575 heartbeater.cc:511] Master 127.3.174.190:45117 requested a full tablet report, sending...
I20260808 04:58:20.643297 4786 ts_manager.cc:194] Registered new tserver with Master: 6d3fc136865448778290a73095b0efd6 (127.3.174.131:45911)
I20260808 04:58:20.643663 4786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:45231
I20260808 04:58:20.648607 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:58:20.655720 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.650322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45168 (local address 127.3.174.190:45117)
0808 04:58:20.650373 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.650375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.650482 (+ 107us) server_negotiation.cc:407] Connection header received
0808 04:58:20.650523 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.650524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.650532 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.650570 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:20.651050 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.651493 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.652219 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.652393 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.652431 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.653610 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:20.653611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:20.653611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:20.653616 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:20.654760 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.655092 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.655093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.655093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.655146 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.655311 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.655311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.655312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.655440 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:20.655510 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.655583 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.655637 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 04:58:20.658103 4786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:20.659969 4786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:20.666607 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.662504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:43615 (local address 127.0.0.1:45368)
0808 04:58:20.662634 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:20.662670 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:20.662731 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:20.663213 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:20.663216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:20.663219 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:20.663284 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.663286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.664301 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.664302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.665417 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.665421 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.666289 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.666291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.666384 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.666400 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 04:58:20.666473 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 04:58:20.666527 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":9,"thread_start_us":55,"threads_started":1}
I20260808 04:58:20.666689 5594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.662708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:42347 (local address 127.0.0.1:52402)
0808 04:58:20.663063 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:20.663073 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:20.663121 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:20.663612 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:20.663613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:20.663615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:20.663667 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.663669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.664858 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.664859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.665655 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.665657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.666441 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.666442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.666496 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.666499 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:20.666563 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 04:58:20.666595 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":50,"threads_started":1}
I20260808 04:58:20.666908 5596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.662843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45368 (local address 127.3.174.130:43615)
0808 04:58:20.663016 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.663020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.663030 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:20.663065 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.663067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.663080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.663153 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:20.663364 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.664202 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.665519 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.666204 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.666280 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.666343 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.666765 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.666807 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260808 04:58:20.667076 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.663080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52402 (local address 127.3.174.129:42347)
0808 04:58:20.663267 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.663270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.663279 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:20.663449 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.663452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.663536 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.663611 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:20.663748 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.664773 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.665726 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.666320 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.666419 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.666472 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.666578 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.666614 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":74,"threads_started":1}
I20260808 04:58:20.667176 5587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.662534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:45911 (local address 127.0.0.1:59972)
0808 04:58:20.662653 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:20.662681 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:20.662738 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:20.663454 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:20.663455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:20.663457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:20.663499 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.663501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.664717 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.664719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.665901 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.665904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.666932 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.666933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.666997 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.667002 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:20.667065 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 04:58:20.667102 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":53,"threads_started":1}
I20260808 04:58:20.667416 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.662628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59972 (local address 127.3.174.131:45911)
0808 04:58:20.663231 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.663236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.663246 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:20.663284 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.663287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.663301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.663375 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:20.663573 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.664610 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.665979 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.666668 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.666923 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.666981 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.667310 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.667351 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":107,"threads_started":1}
I20260808 04:58:20.667694 5022 tablet_service.cc:1467] Processing CreateTablet for tablet 06554112f1f647bba7e90dd8e255e79d (DEFAULT_TABLE table=test-table [id=c29991b91330444e82c93e5d01c50e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:20.668002 5022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06554112f1f647bba7e90dd8e255e79d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:20.669970 5470 tablet_service.cc:1467] Processing CreateTablet for tablet 06554112f1f647bba7e90dd8e255e79d (DEFAULT_TABLE table=test-table [id=c29991b91330444e82c93e5d01c50e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:20.669986 5236 tablet_service.cc:1467] Processing CreateTablet for tablet 06554112f1f647bba7e90dd8e255e79d (DEFAULT_TABLE table=test-table [id=c29991b91330444e82c93e5d01c50e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:20.670291 5470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06554112f1f647bba7e90dd8e255e79d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:20.670320 5236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06554112f1f647bba7e90dd8e255e79d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:20.670732 5602 tablet_bootstrap.cc:492] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0: Bootstrap starting.
I20260808 04:58:20.671500 5602 tablet_bootstrap.cc:654] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:20.672629 5602 log.cc:824] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:20.673574 5602 tablet_bootstrap.cc:492] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0: No bootstrap required, opened a new log
I20260808 04:58:20.673744 5602 ts_tablet_manager.cc:1397] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 04:58:20.674105 5605 tablet_bootstrap.cc:492] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb: Bootstrap starting.
I20260808 04:58:20.675398 5605 tablet_bootstrap.cc:654] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:20.676339 5605 log.cc:824] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:20.676800 5606 tablet_bootstrap.cc:492] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6: Bootstrap starting.
I20260808 04:58:20.677192 5605 tablet_bootstrap.cc:492] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb: No bootstrap required, opened a new log
I20260808 04:58:20.677280 5605 ts_tablet_manager.cc:1397] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 04:58:20.677124 5602 raft_consensus.cc:348] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } }
I20260808 04:58:20.677345 5602 raft_consensus.cc:374] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:20.677386 5602 raft_consensus.cc:729] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6dc0f05a5884966ac848b79dc6d12c0, State: Initialized, Role: FOLLOWER
I20260808 04:58:20.677488 5602 consensus_queue.cc:260] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } }
I20260808 04:58:20.677752 5606 tablet_bootstrap.cc:654] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:20.677913 5602 ts_tablet_manager.cc:1428] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 04:58:20.677925 5126 heartbeater.cc:503] Master 127.3.174.190:45117 was elected leader, sending a full tablet report...
I20260808 04:58:20.678531 5606 log.cc:824] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:20.679575 5606 tablet_bootstrap.cc:492] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6: No bootstrap required, opened a new log
I20260808 04:58:20.679656 5606 ts_tablet_manager.cc:1397] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:58:20.680084 5605 raft_consensus.cc:348] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } }
I20260808 04:58:20.680276 5605 raft_consensus.cc:374] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:20.680327 5605 raft_consensus.cc:729] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d644f6c30b4e5fae88157fba6f4acb, State: Initialized, Role: FOLLOWER
I20260808 04:58:20.680447 5605 consensus_queue.cc:260] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } }
I20260808 04:58:20.680840 5354 heartbeater.cc:503] Master 127.3.174.190:45117 was elected leader, sending a full tablet report...
I20260808 04:58:20.680852 5605 ts_tablet_manager.cc:1428] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260808 04:58:20.683254 5606 raft_consensus.cc:348] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } }
I20260808 04:58:20.683432 5606 raft_consensus.cc:374] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:20.683472 5606 raft_consensus.cc:729] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d3fc136865448778290a73095b0efd6, State: Initialized, Role: FOLLOWER
I20260808 04:58:20.683583 5606 consensus_queue.cc:260] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } }
I20260808 04:58:20.683954 5606 ts_tablet_manager.cc:1428] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260808 04:58:20.683971 5575 heartbeater.cc:503] Master 127.3.174.190:45117 was elected leader, sending a full tablet report...
I20260808 04:58:20.685801 5610 raft_consensus.cc:482] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:20.685875 5610 raft_consensus.cc:504] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } }
I20260808 04:58:20.686299 5610 leader_election.cc:288] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d3fc136865448778290a73095b0efd6 (127.3.174.131:45911), f6dc0f05a5884966ac848b79dc6d12c0 (127.3.174.129:42347)
I20260808 04:58:20.689390 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.686429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:45911 (local address 127.3.174.130:47979)
0808 04:58:20.686481 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:20.686494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:20.686536 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:20.686709 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:20.686710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:20.686712 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:20.686868 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.686870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.687586 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.687587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.688418 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.688421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.689067 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.689069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.689201 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.689205 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:20.689254 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 04:58:20.689319 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 04:58:20.689390 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.686521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:47979 (local address 127.3.174.131:45911)
0808 04:58:20.686566 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.686568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.686576 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:20.686599 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.686599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.686608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.686662 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:20.686934 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.687517 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.688498 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.688969 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.689034 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.689044 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.689270 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.689335 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 04:58:20.689841 5499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06554112f1f647bba7e90dd8e255e79d" candidate_uuid: "81d644f6c30b4e5fae88157fba6f4acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d3fc136865448778290a73095b0efd6" is_pre_election: true
I20260808 04:58:20.690001 5499 raft_consensus.cc:2454] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81d644f6c30b4e5fae88157fba6f4acb in term 0.
I20260808 04:58:20.690021 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.686619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:41275 (local address 127.3.174.129:42347)
0808 04:58:20.686673 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.686675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.686797 (+ 122us) server_negotiation.cc:407] Connection header received
0808 04:58:20.686840 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.686841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.686852 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.686896 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:20.687169 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.687808 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.689022 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.689597 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.689680 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.689690 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.689924 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.689962 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:58:20.690028 5614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.686460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:42347 (local address 127.3.174.130:41275)
0808 04:58:20.686732 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:20.686740 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:20.686792 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:20.686904 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:20.686905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:20.686906 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:20.687084 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.687086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.687873 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.687874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.688943 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:20.688945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.689761 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.689763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:20.689851 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.689855 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:20.689898 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 04:58:20.689927 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":3,"thread_start_us":146,"threads_started":1}
I20260808 04:58:20.690358 5164 leader_election.cc:302] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d3fc136865448778290a73095b0efd6, 81d644f6c30b4e5fae88157fba6f4acb; no voters:
I20260808 04:58:20.690349 5061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06554112f1f647bba7e90dd8e255e79d" candidate_uuid: "81d644f6c30b4e5fae88157fba6f4acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" is_pre_election: true
I20260808 04:58:20.690451 5061 raft_consensus.cc:2454] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81d644f6c30b4e5fae88157fba6f4acb in term 0.
I20260808 04:58:20.690488 5610 raft_consensus.cc:2781] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:20.690539 5610 raft_consensus.cc:482] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:20.690559 5610 raft_consensus.cc:3037] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:20.691700 5610 raft_consensus.cc:504] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } }
I20260808 04:58:20.691888 5610 leader_election.cc:288] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [CANDIDATE]: Term 1 election: Requested vote from peers 6d3fc136865448778290a73095b0efd6 (127.3.174.131:45911), f6dc0f05a5884966ac848b79dc6d12c0 (127.3.174.129:42347)
I20260808 04:58:20.691949 5499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06554112f1f647bba7e90dd8e255e79d" candidate_uuid: "81d644f6c30b4e5fae88157fba6f4acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d3fc136865448778290a73095b0efd6"
I20260808 04:58:20.692041 5499 raft_consensus.cc:3037] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:20.692193 5061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06554112f1f647bba7e90dd8e255e79d" candidate_uuid: "81d644f6c30b4e5fae88157fba6f4acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6dc0f05a5884966ac848b79dc6d12c0"
I20260808 04:58:20.692291 5061 raft_consensus.cc:3037] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:20.693817 5499 raft_consensus.cc:2454] T 06554112f1f647bba7e90dd8e255e79d P 6d3fc136865448778290a73095b0efd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81d644f6c30b4e5fae88157fba6f4acb in term 1.
I20260808 04:58:20.693817 5061 raft_consensus.cc:2454] T 06554112f1f647bba7e90dd8e255e79d P f6dc0f05a5884966ac848b79dc6d12c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81d644f6c30b4e5fae88157fba6f4acb in term 1.
I20260808 04:58:20.693964 5164 leader_election.cc:302] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d3fc136865448778290a73095b0efd6, 81d644f6c30b4e5fae88157fba6f4acb; no voters:
I20260808 04:58:20.694061 5610 raft_consensus.cc:2781] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:20.694198 5610 raft_consensus.cc:686] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [term 1 LEADER]: Becoming Leader. State: Replica: 81d644f6c30b4e5fae88157fba6f4acb, State: Running, Role: LEADER
I20260808 04:58:20.694288 5610 consensus_queue.cc:237] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } }
I20260808 04:58:20.695020 4786 catalog_manager.cc:5495] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb reported cstate change: term changed from 0 to 1, leader changed from <none> to 81d644f6c30b4e5fae88157fba6f4acb (127.3.174.130). New cstate: current_term: 1 leader_uuid: "81d644f6c30b4e5fae88157fba6f4acb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d644f6c30b4e5fae88157fba6f4acb" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d3fc136865448778290a73095b0efd6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6dc0f05a5884966ac848b79dc6d12c0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 42347 } health_report { overall_health: UNKNOWN } } }
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:20.710182 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.702560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.3.174.129:42347)
0808 04:58:20.702609 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.702610 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.702843 (+ 233us) server_negotiation.cc:407] Connection header received
0808 04:58:20.702902 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.702903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.702912 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.702959 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:20.703760 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.704202 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.704848 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.705027 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.705079 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.707431 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:20.707442 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:20.707442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:20.707450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:20.709052 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.709435 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.709439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.709440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.709493 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.709676 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.709676 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.709677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.709815 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:20.709954 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.710015 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.710091 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260808 04:58:20.731101 5355 tablet.cc:2369] T 06554112f1f647bba7e90dd8e255e79d P 81d644f6c30b4e5fae88157fba6f4acb: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:20.735980 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.728587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52414 (local address 127.3.174.129:42347)
0808 04:58:20.728674 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.728676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.728716 (+ 40us) server_negotiation.cc:407] Connection header received
0808 04:58:20.728862 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.728864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.728882 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.728947 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:20.729729 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.730196 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.730905 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.731089 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.731139 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.733512 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:20.733513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:20.733513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:20.733520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:20.734782 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.735102 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.735103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.735103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.735249 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.735487 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.735488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.735488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.735596 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:20.735683 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.735827 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.735901 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[4706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:20.760006 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:20.753493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52424 (local address 127.3.174.129:42347)
0808 04:58:20.753558 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:20.753560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:20.753637 (+ 77us) server_negotiation.cc:407] Connection header received
0808 04:58:20.753716 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:20.753717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:20.753727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:20.753772 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:20.754284 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.754750 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.755429 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:20.755607 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:20.755661 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:20.757762 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:20.757764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:20.757764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:20.757773 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:20.758962 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.759278 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.759279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.759279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.759348 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:20.759515 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:20.759516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:20.759516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:20.759614 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:20.759699 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:20.759822 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 04:58:20.759915 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 04:58:20.760737 4989 tablet_service.cc:2908] Rejecting scan request for tablet 06554112f1f647bba7e90dd8e255e79d: Uninitialized: clean time has not yet been initialized
I20260808 04:58:20.761686 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 4907
I20260808 04:58:20.774976 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 5136
I20260808 04:58:20.789194 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 5362
I20260808 04:58:20.803346 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 4732
2026-08-08T04:58:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2072 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[5643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[5643](info): set up 2 sockets
Aug 08 04:58:20 dist-test-slave-gjw7 krb5kdc[5643](info): commencing operation
krb5kdc: starting...
W20260808 04:58:21.982156 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:22.100368 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:40419
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:40419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:22.182299 5666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:22.182487 5666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:22.182516 5666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:22.183629 5666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:22.183668 5666 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 04:58:22.183686 5666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:22.183702 5666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:22.183718 5666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:22.183741 5666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:22.185189 5666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:40419
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:40419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:22.185453 5666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:22.185658 5666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:58:22.190374 5666 server_base.cc:1034] running on GCE node
W20260808 04:58:22.190348 5677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:22.190348 5675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:22.190442 5672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:22.190860 5666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:22.191131 5666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:22.192291 5666 hybrid_clock.cc:648] HybridClock initialized: now 1786165102192282 us; error 31 us; skew 500 ppm
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:22.194824 5666 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:22.195210 5666 webserver.cc:460] Webserver started at http://127.3.174.190:37077/ using document root <none> and password file <none>
I20260808 04:58:22.195425 5666 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:22.195468 5666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:22.195672 5666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:22.196715 5666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "fa86c3b782924fa09bc452fb3c49dc5d"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "3a53ee9ff1cd28c52073d64e5a9024e9"
server_key_iv: "9e2bfc36233d0c8530e18a4a1f379c94"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.197041 5666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "fa86c3b782924fa09bc452fb3c49dc5d"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "3a53ee9ff1cd28c52073d64e5a9024e9"
server_key_iv: "9e2bfc36233d0c8530e18a4a1f379c94"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.198378 5666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:22.199090 5690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.199258 5666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:22.199342 5666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "fa86c3b782924fa09bc452fb3c49dc5d"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "3a53ee9ff1cd28c52073d64e5a9024e9"
server_key_iv: "9e2bfc36233d0c8530e18a4a1f379c94"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.199406 5666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:22.216275 5666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:22.218183 5666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:22.218312 5666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:22.222846 5666 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:40419
I20260808 04:58:22.222915 5802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:40419 every 8 connection(s)
I20260808 04:58:22.223272 5666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:22.223942 5803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.225051 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 5666
I20260808 04:58:22.225138 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:22.225340 3770 external_mini_cluster.cc:1428] Setting key 1079c4b5dbe702ef0a59fc6470ba0ec3
I20260808 04:58:22.226677 5803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d: Bootstrap starting.
I20260808 04:58:22.227321 5803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.227638 5803 log.cc:824] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:22.228382 5803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d: No bootstrap required, opened a new log
I20260808 04:58:22.230654 5803 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa86c3b782924fa09bc452fb3c49dc5d" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 40419 } }
I20260808 04:58:22.230808 5803 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.230829 5803 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa86c3b782924fa09bc452fb3c49dc5d, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.230945 5803 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa86c3b782924fa09bc452fb3c49dc5d" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 40419 } }
I20260808 04:58:22.231009 5803 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:22.231030 5803 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:22.231076 5803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.231577 5803 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa86c3b782924fa09bc452fb3c49dc5d" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 40419 } }
I20260808 04:58:22.231670 5803 leader_election.cc:302] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa86c3b782924fa09bc452fb3c49dc5d; no voters:
I20260808 04:58:22.231845 5803 leader_election.cc:288] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:22.231878 5809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:22.232048 5809 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [term 1 LEADER]: Becoming Leader. State: Replica: fa86c3b782924fa09bc452fb3c49dc5d, State: Running, Role: LEADER
I20260808 04:58:22.232102 5803 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:22.232179 5809 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa86c3b782924fa09bc452fb3c49dc5d" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 40419 } }
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:22.232574 5809 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa86c3b782924fa09bc452fb3c49dc5d. Latest consensus state: current_term: 1 leader_uuid: "fa86c3b782924fa09bc452fb3c49dc5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa86c3b782924fa09bc452fb3c49dc5d" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 40419 } } }
I20260808 04:58:22.232811 5809 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:22.232740 5812 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa86c3b782924fa09bc452fb3c49dc5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa86c3b782924fa09bc452fb3c49dc5d" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 40419 } } }
I20260808 04:58:22.233281 5812 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:22.233425 5824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:22.233961 5824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:22.235872 5824 catalog_manager.cc:1347] Generated new cluster ID: f9ded712cdda4137ada17272e9ff2fdb
I20260808 04:58:22.235934 5824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:22.236874 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.225903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45502 (local address 127.3.174.190:40419)
0808 04:58:22.226093 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.226104 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.226140 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:58:22.226655 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.226665 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.226729 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.226878 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.227433 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.228716 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.229585 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.230223 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.230344 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.232990 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.232995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.232995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.233007 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.234911 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.235270 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.235274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.235274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.235395 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.235602 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.235603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.235603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.235930 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.236071 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.236522 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.236660 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260808 04:58:22.254575 5824 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:22.255309 5824 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:22.264034 5824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d: Generated new TSK 0
I20260808 04:58:22.264219 5824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:22.296499 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:40419
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:22.377964 5846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:22.378156 5846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:22.378186 5846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:22.379592 5846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:22.379638 5846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:22.379666 5846 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:58:22.379690 5846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:22.381696 5846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.174.190:40419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:22.381886 5846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:22.382071 5846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:22.386970 5857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:22.386940 5856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:22.386930 5859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:22.387243 5846 server_base.cc:1034] running on GCE node
I20260808 04:58:22.387549 5846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:22.387775 5846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:22.388936 5846 hybrid_clock.cc:648] HybridClock initialized: now 1786165102388918 us; error 27 us; skew 500 ppm
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:22.391673 5846 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:22.392050 5846 webserver.cc:460] Webserver started at http://127.3.174.129:38985/ using document root <none> and password file <none>
I20260808 04:58:22.392225 5846 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:22.392263 5846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:22.392355 5846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:22.393400 5846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "7970cadb052f4f9c94afd8ca1fbc0af4"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "68a0d644ba393828d078180a501d2b5a"
server_key_iv: "e9951623890258b45209a5b327798122"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.393704 5846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "7970cadb052f4f9c94afd8ca1fbc0af4"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "68a0d644ba393828d078180a501d2b5a"
server_key_iv: "e9951623890258b45209a5b327798122"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.394965 5846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:22.395594 5868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.395741 5846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:22.395826 5846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "7970cadb052f4f9c94afd8ca1fbc0af4"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "68a0d644ba393828d078180a501d2b5a"
server_key_iv: "e9951623890258b45209a5b327798122"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.395915 5846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:22.407397 5846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:22.409559 5846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:22.409693 5846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:22.410173 5846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:22.410210 5846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.410239 5846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:22.410257 5846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.416530 5846 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:39985
I20260808 04:58:22.416599 6052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:39985 every 8 connection(s)
I20260808 04:58:22.416934 5846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:22.419615 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.410749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:33613 (local address 127.3.174.190:40419)
0808 04:58:22.410797 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.410799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.411043 (+ 244us) server_negotiation.cc:407] Connection header received
0808 04:58:22.411660 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.411662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.411677 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.411747 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.412549 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.413319 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.414769 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.415076 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.415156 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.417536 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.417537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.417538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.417545 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.418621 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.418956 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.418957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.418957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.419001 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.419156 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.419156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.419156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.419249 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.419293 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.419487 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.419552 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 04:58:22.419785 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.410752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:40419 (local address 127.3.174.129:33613)
0808 04:58:22.410920 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.410965 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.411528 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.411834 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.411839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.412164 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:22.412433 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.412440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.413420 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.413421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.414660 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.414664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.415157 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.415158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.415268 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.415784 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:22.415795 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:22.417454 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:22.418683 (+ 1229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.418689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.418691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.418899 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.419037 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.419037 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.419038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.419119 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.419291 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:22.419294 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:22.419396 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.419468 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.419579 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260808 04:58:22.420233 6056 heartbeater.cc:347] Connected to a master server at 127.3.174.190:40419
I20260808 04:58:22.420325 6056 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:22.420513 6056 heartbeater.cc:511] Master 127.3.174.190:40419 requested a full tablet report, sending...
I20260808 04:58:22.421100 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 5846
I20260808 04:58:22.421101 5716 ts_manager.cc:194] Registered new tserver with Master: 7970cadb052f4f9c94afd8ca1fbc0af4 (127.3.174.129:39985)
I20260808 04:58:22.421186 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:22.421446 3770 external_mini_cluster.cc:1428] Setting key 428afc6e90131202fa5232207a370170
I20260808 04:58:22.421983 5716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:33613
I20260808 04:58:22.426666 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.421470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:35263 (local address 127.3.174.190:40419)
0808 04:58:22.421504 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.421506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.421556 (+ 50us) server_negotiation.cc:407] Connection header received
0808 04:58:22.421690 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.421691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.421700 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.421749 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.422102 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.422687 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.423553 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.423737 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.423801 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.424558 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.424560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.424560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.424566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.425726 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.426013 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.426014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.426014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.426067 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.426221 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.426222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.426223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.426327 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.426380 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.426511 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.426578 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 04:58:22.426680 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.421265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:40419 (local address 127.3.174.129:35263)
0808 04:58:22.421498 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.421510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.421568 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.421756 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.421758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.421956 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:22.422014 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.422016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.422805 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.422806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.423470 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.423473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.423811 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.423812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.423890 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.424239 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:22.424242 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:22.424485 (+ 243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:22.425799 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.425800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.425800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.425955 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.426117 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.426118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.426118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.426175 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.426378 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:22.426379 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:22.426424 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.426482 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.426578 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:22.446271 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:40419
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:22.527804 6071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:22.527992 6071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:22.528012 6071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:22.529151 6071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:22.529192 6071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:22.529212 6071 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:58:22.529249 6071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:22.530678 6071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.174.190:40419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:22.530894 6071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:22.531100 6071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:22.535977 6081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:22.536065 6080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:22.536098 6071 server_base.cc:1034] running on GCE node
W20260808 04:58:22.535977 6083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:22.536322 6071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:22.536535 6071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:22.537672 6071 hybrid_clock.cc:648] HybridClock initialized: now 1786165102537655 us; error 26 us; skew 500 ppm
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:22.539968 6071 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:22.540387 6071 webserver.cc:460] Webserver started at http://127.3.174.130:45651/ using document root <none> and password file <none>
I20260808 04:58:22.540601 6071 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:22.540648 6071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:22.540781 6071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:22.541800 6071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "04c381226d6a4cdb995fa379ef50e8b2"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "897b89793f926b2f0fb4bea146213a1e"
server_key_iv: "b31ec20cf9f6ed90f6a3452c42584f45"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.542107 6071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "04c381226d6a4cdb995fa379ef50e8b2"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "897b89793f926b2f0fb4bea146213a1e"
server_key_iv: "b31ec20cf9f6ed90f6a3452c42584f45"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.543273 6071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:22.543915 6096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.544057 6071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:22.544139 6071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "04c381226d6a4cdb995fa379ef50e8b2"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "897b89793f926b2f0fb4bea146213a1e"
server_key_iv: "b31ec20cf9f6ed90f6a3452c42584f45"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.544195 6071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:22.551717 6071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:22.553282 6071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:22.553438 6071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:22.553930 6071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:22.553969 6071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.553996 6071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:22.554020 6071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:22.560880 6071 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:36085
I20260808 04:58:22.560950 6283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:36085 every 8 connection(s)
I20260808 04:58:22.561264 6071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:58:22.563448 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.554262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:43513 (local address 127.3.174.190:40419)
0808 04:58:22.554322 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.554325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.554510 (+ 185us) server_negotiation.cc:407] Connection header received
0808 04:58:22.554946 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.554947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.554961 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.555031 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.555637 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.556403 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.557760 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.558046 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.558122 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.561105 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.561107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.561107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.561114 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.562451 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.562791 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.562792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.562792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.562835 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.562989 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.562990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.562990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.563081 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.563122 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.563317 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.563386 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 04:58:22.563596 6112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.554238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:40419 (local address 127.3.174.130:43513)
0808 04:58:22.554394 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.554433 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.554858 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.555087 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.555091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.555345 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:22.555560 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.555565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.556498 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.556499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.557683 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.557686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.558135 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.558136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.558271 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.558749 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:22.558761 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:22.561003 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:22.562515 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.562518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.562520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.562719 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.562871 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.562871 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.562872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.562955 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.563121 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:22.563124 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:22.563224 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.563301 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.563409 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":79,"threads_started":1}
I20260808 04:58:22.564028 6284 heartbeater.cc:347] Connected to a master server at 127.3.174.190:40419
I20260808 04:58:22.564106 6284 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:22.564314 6284 heartbeater.cc:511] Master 127.3.174.190:40419 requested a full tablet report, sending...
I20260808 04:58:22.564682 5718 ts_manager.cc:194] Registered new tserver with Master: 04c381226d6a4cdb995fa379ef50e8b2 (127.3.174.130:36085)
I20260808 04:58:22.565150 5718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:43513
I20260808 04:58:22.569784 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.564990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:34535 (local address 127.3.174.190:40419)
0808 04:58:22.565024 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.565026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.565128 (+ 102us) server_negotiation.cc:407] Connection header received
0808 04:58:22.565184 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.565185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.565194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.565245 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.565614 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.566068 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.566722 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.566923 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.566981 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.567757 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.567759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.567759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.567766 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.568776 (+ 1010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.569057 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.569058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.569058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.569110 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.569268 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.569269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.569269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.569435 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.569495 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.569633 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.569694 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 04:58:22.569799 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.564941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:40419 (local address 127.3.174.130:34535)
0808 04:58:22.565067 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.565077 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.565131 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.565249 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.565250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.565472 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:22.565545 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.565547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.566153 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.566154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.566654 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.566656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.566985 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.566986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.567059 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.567415 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:22.567420 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:22.567680 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:22.568845 (+ 1165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.568846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.568846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.568996 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.569162 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.569163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.569163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.569218 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.569495 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:22.569496 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:22.569541 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.569600 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.569694 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260808 04:58:22.570852 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 6071
I20260808 04:58:22.570963 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:22.571153 3770 external_mini_cluster.cc:1428] Setting key a351a35315b84105259e948b6c0b1034
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:22.594259 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:40419
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:22.675007 6298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:22.675202 6298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:22.675235 6298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:22.676357 6298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:22.676396 6298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:22.676424 6298 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:58:22.676446 6298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:22.677871 6298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.174.190:40419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:22.678097 6298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:22.678305 6298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:22.682880 6304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:22.683038 6298 server_base.cc:1034] running on GCE node
W20260808 04:58:22.682952 6310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:22.683094 6306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:22.683310 6298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:22.683516 6298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:22.684654 6298 hybrid_clock.cc:648] HybridClock initialized: now 1786165102684641 us; error 27 us; skew 500 ppm
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:22.687014 6298 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:22.687423 6298 webserver.cc:460] Webserver started at http://127.3.174.131:37225/ using document root <none> and password file <none>
I20260808 04:58:22.687631 6298 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:22.687677 6298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:22.687803 6298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:22.689040 6298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "911b250ba07645e18eaeae4467fb2df8"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "f94556b8c41263855e59abd11807a66b"
server_key_iv: "09e3c905186f45c5a104009f7b2aea28"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.689440 6298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "911b250ba07645e18eaeae4467fb2df8"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "f94556b8c41263855e59abd11807a66b"
server_key_iv: "09e3c905186f45c5a104009f7b2aea28"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.690594 6298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:22.691258 6318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.691424 6298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:22.691504 6298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "911b250ba07645e18eaeae4467fb2df8"
format_stamp: "Formatted at 2026-08-08 04:58:22 on dist-test-slave-gjw7"
server_key: "f94556b8c41263855e59abd11807a66b"
server_key_iv: "09e3c905186f45c5a104009f7b2aea28"
server_key_version: "encryptionkey@0"
I20260808 04:58:22.691570 6298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:22.700412 6298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:22.702557 6298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:22.702693 6298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:22.703199 6298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:22.703238 6298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.703267 6298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:22.703306 6298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:22.710405 6298 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:40791
I20260808 04:58:22.710464 6485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:40791 every 8 connection(s)
I20260808 04:58:22.710811 6298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 04:58:22.711663 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.703582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:53961 (local address 127.3.174.190:40419)
0808 04:58:22.703638 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.703641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.703830 (+ 189us) server_negotiation.cc:407] Connection header received
0808 04:58:22.704291 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.704293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.704306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.704355 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.704959 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.705535 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.706469 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.706681 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.706739 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.708769 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.708771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.708771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.708778 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.710362 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.710739 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.710741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.710741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.710805 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.711054 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.711055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.711055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.711196 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.711266 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.711497 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.711573 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 04:58:22.711799 6329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.703574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:40419 (local address 127.3.174.131:53961)
0808 04:58:22.703713 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.703751 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.704198 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.704424 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.704427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.704683 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:22.704859 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.704864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.705615 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.705616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.706388 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.706391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.706750 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.706751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.706865 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.707294 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:22.707302 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:22.708682 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:22.710435 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.710439 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.710441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.710670 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.710888 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.710889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.710890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.710989 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.711258 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:22.711263 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:22.711380 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.711465 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.711607 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260808 04:58:22.712388 6486 heartbeater.cc:347] Connected to a master server at 127.3.174.190:40419
I20260808 04:58:22.712471 6486 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:22.712639 6486 heartbeater.cc:511] Master 127.3.174.190:40419 requested a full tablet report, sending...
I20260808 04:58:22.713006 5718 ts_manager.cc:194] Registered new tserver with Master: 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791)
I20260808 04:58:22.713428 5718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:53961
I20260808 04:58:22.718081 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.713352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:49499 (local address 127.3.174.190:40419)
0808 04:58:22.713389 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.713391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.713563 (+ 172us) server_negotiation.cc:407] Connection header received
0808 04:58:22.713599 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.713600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.713609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.713648 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.713932 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.714381 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.715034 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.715225 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.715277 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.716139 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.716140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.716140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.716147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.717077 (+ 930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.717391 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.717392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.717393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.717443 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.717603 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.717603 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.717604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.717702 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.717755 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.717936 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.717996 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 04:58:22.718097 6495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.713266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:40419 (local address 127.3.174.131:49499)
0808 04:58:22.713494 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.713502 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.713553 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.713651 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.713652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.713817 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:22.713862 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.713863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.714462 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.714463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.714966 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.714967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.715279 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.715280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.715357 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.715738 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:22.715743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:22.716063 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:22.717146 (+ 1083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.717147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.717147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.717331 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.717495 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.717496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.717496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.717553 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.717753 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:22.717754 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:22.717838 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.717900 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.717996 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":184,"threads_started":1}
I20260808 04:58:22.718601 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 6298
I20260808 04:58:22.718688 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:22.718847 3770 external_mini_cluster.cc:1428] Setting key d36f7c92ee3849af747381fb322d8c41
I20260808 04:58:22.719813 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:22.742376 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.736328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45512 (local address 127.3.174.190:40419)
0808 04:58:22.736396 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.736398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.736412 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:58:22.736468 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.736468 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.736477 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.736523 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.737056 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.737585 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.738241 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.738410 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.738451 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.740337 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.740338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.740339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.740346 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.741476 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.741835 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.741835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.741836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.741880 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.742014 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.742014 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.742014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.742106 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.742170 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.742256 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.742312 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 04:58:22.743885 5718 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:22.745538 5718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:22.752015 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.747687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:36085 (local address 127.0.0.1:56274)
0808 04:58:22.748357 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.748368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.748426 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.748666 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.748669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.748671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:22.748753 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.748755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.749785 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.749786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.750985 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.750988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.751725 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.751726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.751806 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.751820 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.751883 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.751936 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":5,"thread_start_us":80,"threads_started":1}
I20260808 04:58:22.752054 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.747872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56274 (local address 127.3.174.130:36085)
0808 04:58:22.748312 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.748316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.748440 (+ 124us) server_negotiation.cc:407] Connection header received
0808 04:58:22.748495 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.748498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.748516 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.748610 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:22.748829 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.749700 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.751092 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.751632 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.751714 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.751778 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.751918 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.751954 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":84,"threads_started":1}
I20260808 04:58:22.752264 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.747689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39985 (local address 127.0.0.1:43668)
0808 04:58:22.747986 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.748029 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.748099 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.748731 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.748732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.748735 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:22.748794 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.748796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.750073 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.750074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.751117 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.751120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.752040 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.752041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.752105 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.752109 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.752140 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.752178 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":60,"threads_started":1}
I20260808 04:58:22.752393 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.747899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:40791 (local address 127.0.0.1:56282)
0808 04:58:22.748748 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.748758 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.748808 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.749167 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.749168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.749171 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:22.749210 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.749212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.750333 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.750334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.751140 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.751143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.752099 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.752099 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.752195 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.752198 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.752240 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.752269 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":2,"thread_start_us":52,"threads_started":1}
I20260808 04:58:22.752450 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.748053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.3.174.131:40791)
0808 04:58:22.748265 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.748270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.748874 (+ 604us) server_negotiation.cc:407] Connection header received
0808 04:58:22.748918 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.748921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.748937 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.749164 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:22.749282 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.750251 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.751232 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.751937 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.752015 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.752069 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.752310 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.752350 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260808 04:58:22.752566 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.747807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43668 (local address 127.3.174.129:39985)
0808 04:58:22.748530 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.748534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.748544 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:22.748583 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.748585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.748600 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.748671 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:22.748871 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.749985 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.751204 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.751937 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.752346 (+ 409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.752408 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.752435 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.752476 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":82,"threads_started":1}
I20260808 04:58:22.753226 5946 tablet_service.cc:1467] Processing CreateTablet for tablet 25c5e6caa1fd4683b1c1e3f8b10936d8 (DEFAULT_TABLE table=test-table [id=2c3a5d0b6512448aa4cf37bf2200e079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:22.753594 5946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c5e6caa1fd4683b1c1e3f8b10936d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.753481 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 25c5e6caa1fd4683b1c1e3f8b10936d8 (DEFAULT_TABLE table=test-table [id=2c3a5d0b6512448aa4cf37bf2200e079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:22.753814 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c5e6caa1fd4683b1c1e3f8b10936d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.756585 6519 tablet_bootstrap.cc:492] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: Bootstrap starting.
I20260808 04:58:22.756851 6385 tablet_service.cc:1467] Processing CreateTablet for tablet 25c5e6caa1fd4683b1c1e3f8b10936d8 (DEFAULT_TABLE table=test-table [id=2c3a5d0b6512448aa4cf37bf2200e079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:22.757179 6385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c5e6caa1fd4683b1c1e3f8b10936d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.757437 6523 tablet_bootstrap.cc:492] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: Bootstrap starting.
I20260808 04:58:22.757462 6519 tablet_bootstrap.cc:654] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.757784 6519 log.cc:824] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:22.758096 6523 tablet_bootstrap.cc:654] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.758503 6523 log.cc:824] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:22.758701 6519 tablet_bootstrap.cc:492] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: No bootstrap required, opened a new log
I20260808 04:58:22.758777 6519 ts_tablet_manager.cc:1397] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 04:58:22.759260 6523 tablet_bootstrap.cc:492] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: No bootstrap required, opened a new log
I20260808 04:58:22.759384 6523 ts_tablet_manager.cc:1397] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:58:22.760260 6530 tablet_bootstrap.cc:492] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: Bootstrap starting.
I20260808 04:58:22.760905 6519 raft_consensus.cc:348] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.761039 6519 raft_consensus.cc:374] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.761070 6519 raft_consensus.cc:729] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7970cadb052f4f9c94afd8ca1fbc0af4, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.761206 6519 consensus_queue.cc:260] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.761246 6530 tablet_bootstrap.cc:654] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.761509 6519 ts_tablet_manager.cc:1428] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 04:58:22.761605 6056 heartbeater.cc:503] Master 127.3.174.190:40419 was elected leader, sending a full tablet report...
I20260808 04:58:22.761633 6530 log.cc:824] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:22.762750 6530 tablet_bootstrap.cc:492] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: No bootstrap required, opened a new log
I20260808 04:58:22.762828 6530 ts_tablet_manager.cc:1397] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 04:58:22.762821 6523 raft_consensus.cc:348] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.762984 6523 raft_consensus.cc:374] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.763031 6523 raft_consensus.cc:729] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c381226d6a4cdb995fa379ef50e8b2, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.763151 6523 consensus_queue.cc:260] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.763535 6523 ts_tablet_manager.cc:1428] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 04:58:22.763573 6284 heartbeater.cc:503] Master 127.3.174.190:40419 was elected leader, sending a full tablet report...
I20260808 04:58:22.766566 6530 raft_consensus.cc:348] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.766695 6530 raft_consensus.cc:374] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.766716 6530 raft_consensus.cc:729] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 911b250ba07645e18eaeae4467fb2df8, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.766809 6530 consensus_queue.cc:260] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.767050 6530 ts_tablet_manager.cc:1428] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: Time spent starting tablet: real 0.004s user 0.003s sys 0.002s
I20260808 04:58:22.767095 6486 heartbeater.cc:503] Master 127.3.174.190:40419 was elected leader, sending a full tablet report...
I20260808 04:58:22.794513 6538 raft_consensus.cc:482] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:22.794636 6538 raft_consensus.cc:504] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.795064 6538 leader_election.cc:288] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04c381226d6a4cdb995fa379ef50e8b2 (127.3.174.130:36085), 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791)
I20260808 04:58:22.798966 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.795349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:40791 (local address 127.3.174.129:35579)
0808 04:58:22.795397 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.795409 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.795457 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.795601 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.795602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.795604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:22.795776 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.795778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.796724 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.796725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.797966 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.797969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.798706 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.798707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.798818 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.798822 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.798861 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.798893 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 04:58:22.799014 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.795414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:35579 (local address 127.3.174.131:40791)
0808 04:58:22.795466 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.795468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.795480 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:58:22.795510 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.795511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.795524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.795593 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:22.795848 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.796634 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.798076 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.798632 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.798706 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.798716 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.798901 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.798945 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 04:58:22.799388 6414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c5e6caa1fd4683b1c1e3f8b10936d8" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911b250ba07645e18eaeae4467fb2df8" is_pre_election: true
I20260808 04:58:22.799567 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.795343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:53111 (local address 127.3.174.130:36085)
0808 04:58:22.795396 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.795398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.795651 (+ 253us) server_negotiation.cc:407] Connection header received
0808 04:58:22.795696 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.795697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.795705 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.795753 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:22.796027 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.796713 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.798311 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.799021 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.799093 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.799104 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.799464 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.799508 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 04:58:22.799562 6414 raft_consensus.cc:2454] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 0.
I20260808 04:58:22.799695 6543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.795351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:36085 (local address 127.3.174.129:53111)
0808 04:58:22.795577 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.795586 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.795639 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.795768 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.795769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.795771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:22.795952 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.795954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.796785 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.796786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.798221 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.798224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.799235 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.799236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.799383 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.799388 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.799437 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.799467 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":13,"thread_start_us":160,"threads_started":1}
I20260808 04:58:22.799964 5870 leader_election.cc:302] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7970cadb052f4f9c94afd8ca1fbc0af4, 911b250ba07645e18eaeae4467fb2df8; no voters:
I20260808 04:58:22.799999 6541 raft_consensus.cc:482] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:22.800097 6541 raft_consensus.cc:504] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.800197 6538 raft_consensus.cc:2781] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:22.800177 6213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c5e6caa1fd4683b1c1e3f8b10936d8" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c381226d6a4cdb995fa379ef50e8b2" is_pre_election: true
I20260808 04:58:22.800251 6538 raft_consensus.cc:482] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:22.800275 6538 raft_consensus.cc:3037] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.800276 6213 raft_consensus.cc:2454] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 0.
I20260808 04:58:22.800433 6541 leader_election.cc:288] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7970cadb052f4f9c94afd8ca1fbc0af4 (127.3.174.129:39985), 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791)
I20260808 04:58:22.802358 6538 raft_consensus.cc:504] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.802492 6538 leader_election.cc:288] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 election: Requested vote from peers 04c381226d6a4cdb995fa379ef50e8b2 (127.3.174.130:36085), 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791)
I20260808 04:58:22.802735 6414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c5e6caa1fd4683b1c1e3f8b10936d8" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911b250ba07645e18eaeae4467fb2df8"
I20260808 04:58:22.802755 6213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c5e6caa1fd4683b1c1e3f8b10936d8" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c381226d6a4cdb995fa379ef50e8b2"
I20260808 04:58:22.802829 6213 raft_consensus.cc:3037] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.802829 6414 raft_consensus.cc:3037] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.803771 6112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.800580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39985 (local address 127.3.174.130:44517)
0808 04:58:22.800625 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.800636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.800676 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.800815 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.800816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.800818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:22.800968 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.800970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.801795 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.801797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.802685 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.802689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.803480 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.803481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.803586 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.803590 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.803652 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.803702 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 04:58:22.804059 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.800638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:44517 (local address 127.3.174.129:39985)
0808 04:58:22.800677 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.800679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.800710 (+ 31us) server_negotiation.cc:407] Connection header received
0808 04:58:22.800734 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.800735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.800744 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.800801 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:22.801056 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.801715 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.802825 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.803353 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.803432 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.803441 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.803955 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.803994 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 04:58:22.804484 6414 raft_consensus.cc:2454] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 1.
I20260808 04:58:22.804577 6213 raft_consensus.cc:2454] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 1.
I20260808 04:58:22.804610 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.800962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:55597 (local address 127.3.174.131:40791)
0808 04:58:22.800998 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.801000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.801007 (+ 7us) server_negotiation.cc:407] Connection header received
0808 04:58:22.801063 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.801064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.801072 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.801127 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:22.801324 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.801815 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.803473 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.803982 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.804041 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.804049 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.804501 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.804553 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 04:58:22.804832 5978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c5e6caa1fd4683b1c1e3f8b10936d8" candidate_uuid: "04c381226d6a4cdb995fa379ef50e8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" is_pre_election: true
I20260808 04:58:22.804955 5978 raft_consensus.cc:2379] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 04c381226d6a4cdb995fa379ef50e8b2 in current term 1: Already voted for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in this term.
I20260808 04:58:22.805028 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.800704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:40791 (local address 127.3.174.130:55597)
0808 04:58:22.800921 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.800931 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.801010 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.801132 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.801132 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.801134 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:22.801242 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.801244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.801900 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.801901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.803367 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.803370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.804232 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.804234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.804405 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.804410 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.804865 (+ 455us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.804907 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":165,"threads_started":1}
I20260808 04:58:22.805181 5870 leader_election.cc:302] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7970cadb052f4f9c94afd8ca1fbc0af4, 911b250ba07645e18eaeae4467fb2df8; no voters:
I20260808 04:58:22.805325 6538 raft_consensus.cc:2781] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:22.805341 6414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c5e6caa1fd4683b1c1e3f8b10936d8" candidate_uuid: "04c381226d6a4cdb995fa379ef50e8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911b250ba07645e18eaeae4467fb2df8" is_pre_election: true
I20260808 04:58:22.805425 6414 raft_consensus.cc:2379] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 04c381226d6a4cdb995fa379ef50e8b2 in current term 1: Already voted for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in this term.
I20260808 04:58:22.805498 6538 raft_consensus.cc:686] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 LEADER]: Becoming Leader. State: Replica: 7970cadb052f4f9c94afd8ca1fbc0af4, State: Running, Role: LEADER
I20260808 04:58:22.805595 6538 consensus_queue.cc:237] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.805675 6099 leader_election.cc:302] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04c381226d6a4cdb995fa379ef50e8b2; no voters: 7970cadb052f4f9c94afd8ca1fbc0af4, 911b250ba07645e18eaeae4467fb2df8
I20260808 04:58:22.805860 6541 raft_consensus.cc:2726] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 04:58:22.806494 5718 catalog_manager.cc:5495] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7970cadb052f4f9c94afd8ca1fbc0af4 (127.3.174.129). New cstate: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } health_report { overall_health: UNKNOWN } } }
W20260808 04:58:22.811822 6288 tablet.cc:2369] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:22.824523 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.822317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.3.174.129:39985)
0808 04:58:22.822365 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.822367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.822391 (+ 24us) server_negotiation.cc:407] Connection header received
0808 04:58:22.822472 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.822473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.822487 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.822542 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:22.822753 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.823176 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.823922 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.824094 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.824143 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.824283 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:22.824370 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:22.824424 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.824452 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.824466 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 04:58:22.826543 6414 raft_consensus.cc:1261] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Refusing update from remote peer 7970cadb052f4f9c94afd8ca1fbc0af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:22.826555 6213 raft_consensus.cc:1261] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Refusing update from remote peer 7970cadb052f4f9c94afd8ca1fbc0af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:22.826874 6545 consensus_queue.cc:1035] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:22.826997 6538 consensus_queue.cc:1035] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:22.830554 6553 mvcc.cc:205] Tried to move back new op lower bound from 7316132261175070720 to 7316132261092450304. Current Snapshot: MvccSnapshot[applied={T|T < 7316132261175070720 or (T in {7316132261175070720})}]
I20260808 04:58:22.830715 6552 mvcc.cc:205] Tried to move back new op lower bound from 7316132261175070720 to 7316132261092450304. Current Snapshot: MvccSnapshot[applied={T|T < 7316132261175070720 or (T in {7316132261175070720})}]
I20260808 04:58:22.833937 5716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:22.834055 5716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:22.834851 5716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d: Sending DeleteTablet for 3 replicas of tablet 25c5e6caa1fd4683b1c1e3f8b10936d8
I20260808 04:58:22.835242 6180 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c5e6caa1fd4683b1c1e3f8b10936d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:22 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:56274
I20260808 04:58:22.835558 6564 tablet_replica.cc:330] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: stopping tablet replica
I20260808 04:58:22.835587 6385 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c5e6caa1fd4683b1c1e3f8b10936d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:22 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:56282
I20260808 04:58:22.835723 6564 raft_consensus.cc:2229] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:22.835893 6564 raft_consensus.cc:2258] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:22.836082 5716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:22.836230 5716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:22.836841 6566 tablet_replica.cc:330] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: stopping tablet replica
I20260808 04:58:22.836941 6566 raft_consensus.cc:2229] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:22.837100 6566 raft_consensus.cc:2258] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:22.837597 6566 ts_tablet_manager.cc:1905] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:22.837864 5946 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c5e6caa1fd4683b1c1e3f8b10936d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:22 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:43668
I20260808 04:58:22.838306 6568 tablet_replica.cc:330] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: stopping tablet replica
I20260808 04:58:22.838444 6568 raft_consensus.cc:2229] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:22.838590 6568 raft_consensus.cc:2258] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:22.839058 6568 ts_tablet_manager.cc:1905] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:22.839569 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 84c340885cdd4d9d8fbd3022053e7474 (DEFAULT_TABLE table=test-table [id=dc8f5f86a0fd4571abeeb694129b24af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:22.839697 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c340885cdd4d9d8fbd3022053e7474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.839823 6566 ts_tablet_manager.cc:1918] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:22.839953 6566 log.cc:1196] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/25c5e6caa1fd4683b1c1e3f8b10936d8
I20260808 04:58:22.840273 6566 ts_tablet_manager.cc:1939] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 911b250ba07645e18eaeae4467fb2df8: Deleting consensus metadata
I20260808 04:58:22.841480 6523 tablet_bootstrap.cc:492] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: Bootstrap starting.
I20260808 04:58:22.841584 6568 ts_tablet_manager.cc:1918] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:22.841557 5693 catalog_manager.cc:4841] TS 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791): tablet 25c5e6caa1fd4683b1c1e3f8b10936d8 (table test-table [id=2c3a5d0b6512448aa4cf37bf2200e079]) successfully deleted
I20260808 04:58:22.841646 6568 log.cc:1196] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/25c5e6caa1fd4683b1c1e3f8b10936d8
I20260808 04:58:22.841876 6385 tablet_service.cc:1467] Processing CreateTablet for tablet 84c340885cdd4d9d8fbd3022053e7474 (DEFAULT_TABLE table=test-table [id=dc8f5f86a0fd4571abeeb694129b24af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:22.841915 6568 ts_tablet_manager.cc:1939] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 7970cadb052f4f9c94afd8ca1fbc0af4: Deleting consensus metadata
I20260808 04:58:22.841984 6385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c340885cdd4d9d8fbd3022053e7474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.842494 6523 tablet_bootstrap.cc:654] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.843236 6530 tablet_bootstrap.cc:492] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: Bootstrap starting.
I20260808 04:58:22.843554 5693 catalog_manager.cc:4841] TS 7970cadb052f4f9c94afd8ca1fbc0af4 (127.3.174.129:39985): tablet 25c5e6caa1fd4683b1c1e3f8b10936d8 (table test-table [id=2c3a5d0b6512448aa4cf37bf2200e079]) successfully deleted
I20260808 04:58:22.843894 6564 ts_tablet_manager.cc:1905] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:22.844297 6530 tablet_bootstrap.cc:654] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.845253 6523 tablet_bootstrap.cc:492] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: No bootstrap required, opened a new log
I20260808 04:58:22.845314 6523 ts_tablet_manager.cc:1397] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 04:58:22.845578 6530 tablet_bootstrap.cc:492] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: No bootstrap required, opened a new log
I20260808 04:58:22.845624 6530 ts_tablet_manager.cc:1397] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 04:58:22.845803 6523 raft_consensus.cc:348] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.845876 6523 raft_consensus.cc:374] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.845850 6530 raft_consensus.cc:348] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.845896 6523 raft_consensus.cc:729] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c381226d6a4cdb995fa379ef50e8b2, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.845907 6530 raft_consensus.cc:374] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.845925 6530 raft_consensus.cc:729] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 911b250ba07645e18eaeae4467fb2df8, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.845938 6523 consensus_queue.cc:260] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.845960 6530 consensus_queue.cc:260] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.846019 6523 ts_tablet_manager.cc:1428] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:22.846046 6530 ts_tablet_manager.cc:1428] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.846136 6564 ts_tablet_manager.cc:1918] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:22.846208 6564 log.cc:1196] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/25c5e6caa1fd4683b1c1e3f8b10936d8
I20260808 04:58:22.846443 6564 ts_tablet_manager.cc:1939] T 25c5e6caa1fd4683b1c1e3f8b10936d8 P 04c381226d6a4cdb995fa379ef50e8b2: Deleting consensus metadata
I20260808 04:58:22.846582 5946 tablet_service.cc:1467] Processing CreateTablet for tablet 84c340885cdd4d9d8fbd3022053e7474 (DEFAULT_TABLE table=test-table [id=dc8f5f86a0fd4571abeeb694129b24af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:22.846685 5946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c340885cdd4d9d8fbd3022053e7474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.846841 5692 catalog_manager.cc:4841] TS 04c381226d6a4cdb995fa379ef50e8b2 (127.3.174.130:36085): tablet 25c5e6caa1fd4683b1c1e3f8b10936d8 (table test-table [id=2c3a5d0b6512448aa4cf37bf2200e079]) successfully deleted
I20260808 04:58:22.848222 6519 tablet_bootstrap.cc:492] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: Bootstrap starting.
I20260808 04:58:22.849159 6519 tablet_bootstrap.cc:654] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.850275 6519 tablet_bootstrap.cc:492] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: No bootstrap required, opened a new log
I20260808 04:58:22.850318 6519 ts_tablet_manager.cc:1397] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 04:58:22.850772 6519 raft_consensus.cc:348] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.850841 6519 raft_consensus.cc:374] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.850860 6519 raft_consensus.cc:729] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7970cadb052f4f9c94afd8ca1fbc0af4, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.850898 6519 consensus_queue.cc:260] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.850982 6519 ts_tablet_manager.cc:1428] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:22.875275 6545 raft_consensus.cc:482] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:22.875365 6545 raft_consensus.cc:504] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.875530 6545 leader_election.cc:288] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04c381226d6a4cdb995fa379ef50e8b2 (127.3.174.130:36085), 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791)
I20260808 04:58:22.875744 6414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c340885cdd4d9d8fbd3022053e7474" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911b250ba07645e18eaeae4467fb2df8" is_pre_election: true
I20260808 04:58:22.875747 6213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c340885cdd4d9d8fbd3022053e7474" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c381226d6a4cdb995fa379ef50e8b2" is_pre_election: true
I20260808 04:58:22.875836 6414 raft_consensus.cc:2454] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 0.
I20260808 04:58:22.875846 6213 raft_consensus.cc:2454] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 0.
I20260808 04:58:22.875993 5870 leader_election.cc:302] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7970cadb052f4f9c94afd8ca1fbc0af4, 911b250ba07645e18eaeae4467fb2df8; no voters:
I20260808 04:58:22.876102 6545 raft_consensus.cc:2781] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:22.876154 6545 raft_consensus.cc:482] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:22.876175 6545 raft_consensus.cc:3037] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.877483 6545 raft_consensus.cc:504] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.877606 6545 leader_election.cc:288] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 election: Requested vote from peers 04c381226d6a4cdb995fa379ef50e8b2 (127.3.174.130:36085), 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791)
I20260808 04:58:22.877781 6213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c340885cdd4d9d8fbd3022053e7474" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c381226d6a4cdb995fa379ef50e8b2"
I20260808 04:58:22.877799 6414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c340885cdd4d9d8fbd3022053e7474" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911b250ba07645e18eaeae4467fb2df8"
I20260808 04:58:22.877872 6414 raft_consensus.cc:3037] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.877875 6213 raft_consensus.cc:3037] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.878748 6414 raft_consensus.cc:2454] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 1.
I20260808 04:58:22.878837 6213 raft_consensus.cc:2454] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 1.
I20260808 04:58:22.878911 5870 leader_election.cc:302] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7970cadb052f4f9c94afd8ca1fbc0af4, 911b250ba07645e18eaeae4467fb2df8; no voters:
I20260808 04:58:22.878988 6545 raft_consensus.cc:2781] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:22.879032 6545 raft_consensus.cc:686] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 LEADER]: Becoming Leader. State: Replica: 7970cadb052f4f9c94afd8ca1fbc0af4, State: Running, Role: LEADER
I20260808 04:58:22.879076 6545 consensus_queue.cc:237] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.879670 5716 catalog_manager.cc:5495] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7970cadb052f4f9c94afd8ca1fbc0af4 (127.3.174.129). New cstate: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } health_report { overall_health: UNKNOWN } } }
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:22.913443 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.906101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45526 (local address 127.3.174.190:40419)
0808 04:58:22.906164 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.906166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.906359 (+ 193us) server_negotiation.cc:407] Connection header received
0808 04:58:22.906411 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.906412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.906424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.906477 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.906999 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.907759 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.908463 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.908761 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.908838 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.911052 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.911053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.911053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.911060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.912337 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.912737 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.912737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.912738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.912782 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.912961 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.912962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.912962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.913062 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.913104 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.913268 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.913378 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":37}
I20260808 04:58:22.913560 6583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.906017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:40419 (local address 127.0.0.1:45526)
0808 04:58:22.906286 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.906296 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.906353 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.906494 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.906495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.906862 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:22.906910 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.906912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.907852 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.907853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.908381 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.908382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.908832 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.908832 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.908867 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.909443 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:22.909448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:22.910957 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:22.912404 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.912412 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.912415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.912662 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.912834 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:22.912835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:22.912835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:22.912906 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:22.913109 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:22.913117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:22.913170 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.913243 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.913449 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":53,"thread_start_us":143,"threads_started":1}
I20260808 04:58:22.915760 5716 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:45526:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 04:58:22.916038 5716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:22.917510 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 7d8e61a6af89467c99ae0f144006529e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07e5d81dbaa6492893a019b9c4ec08dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:58:22.917582 5946 tablet_service.cc:1467] Processing CreateTablet for tablet 7d8e61a6af89467c99ae0f144006529e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07e5d81dbaa6492893a019b9c4ec08dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:58:22.917606 6385 tablet_service.cc:1467] Processing CreateTablet for tablet 7d8e61a6af89467c99ae0f144006529e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07e5d81dbaa6492893a019b9c4ec08dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
W20260808 04:58:22.917652 6057 tablet.cc:2369] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:22.917735 6385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d8e61a6af89467c99ae0f144006529e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.917737 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d8e61a6af89467c99ae0f144006529e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.917738 5946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d8e61a6af89467c99ae0f144006529e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:22.919541 6519 tablet_bootstrap.cc:492] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: Bootstrap starting.
I20260808 04:58:22.919538 6523 tablet_bootstrap.cc:492] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: Bootstrap starting.
I20260808 04:58:22.920753 6523 tablet_bootstrap.cc:654] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.920754 6519 tablet_bootstrap.cc:654] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.921670 6530 tablet_bootstrap.cc:492] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: Bootstrap starting.
I20260808 04:58:22.922006 6519 tablet_bootstrap.cc:492] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: No bootstrap required, opened a new log
I20260808 04:58:22.922060 6519 ts_tablet_manager.cc:1397] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 04:58:22.922247 6519 raft_consensus.cc:348] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.922319 6519 raft_consensus.cc:374] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.922339 6519 raft_consensus.cc:729] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7970cadb052f4f9c94afd8ca1fbc0af4, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.922381 6519 consensus_queue.cc:260] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.922466 6519 ts_tablet_manager.cc:1428] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.922575 6545 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.922639 6545 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:22.922866 6530 tablet_bootstrap.cc:654] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:22.924263 6530 tablet_bootstrap.cc:492] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: No bootstrap required, opened a new log
I20260808 04:58:22.924302 6530 ts_tablet_manager.cc:1397] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 04:58:22.924624 6530 raft_consensus.cc:348] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.924685 6530 raft_consensus.cc:374] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.924700 6530 raft_consensus.cc:729] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 911b250ba07645e18eaeae4467fb2df8, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.924732 6530 consensus_queue.cc:260] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.924813 6530 ts_tablet_manager.cc:1428] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:22.924923 6542 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.925055 6523 tablet_bootstrap.cc:492] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: No bootstrap required, opened a new log
I20260808 04:58:22.925073 6542 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:22.925096 6523 ts_tablet_manager.cc:1397] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260808 04:58:22.925489 6523 raft_consensus.cc:348] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.925568 6523 raft_consensus.cc:374] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:22.925590 6523 raft_consensus.cc:729] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c381226d6a4cdb995fa379ef50e8b2, State: Initialized, Role: FOLLOWER
I20260808 04:58:22.925626 6523 consensus_queue.cc:260] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.925711 6523 ts_tablet_manager.cc:1428] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:22.925755 6541 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.925844 6541 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:22.931595 6545 raft_consensus.cc:482] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:22.931661 6545 raft_consensus.cc:504] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.931800 6545 leader_election.cc:288] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04c381226d6a4cdb995fa379ef50e8b2 (127.3.174.130:36085), 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791)
I20260808 04:58:22.931993 6213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8e61a6af89467c99ae0f144006529e" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c381226d6a4cdb995fa379ef50e8b2" is_pre_election: true
I20260808 04:58:22.932067 6213 raft_consensus.cc:2454] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 0.
I20260808 04:58:22.931988 6414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8e61a6af89467c99ae0f144006529e" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911b250ba07645e18eaeae4467fb2df8" is_pre_election: true
I20260808 04:58:22.932143 6414 raft_consensus.cc:2454] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 0.
I20260808 04:58:22.932209 5869 leader_election.cc:302] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c381226d6a4cdb995fa379ef50e8b2, 7970cadb052f4f9c94afd8ca1fbc0af4; no voters:
I20260808 04:58:22.932281 6545 raft_consensus.cc:2781] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:22.932323 6545 raft_consensus.cc:482] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:22.932343 6545 raft_consensus.cc:3037] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.933243 6545 raft_consensus.cc:504] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.933383 6545 leader_election.cc:288] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 election: Requested vote from peers 04c381226d6a4cdb995fa379ef50e8b2 (127.3.174.130:36085), 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791)
I20260808 04:58:22.933503 6213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8e61a6af89467c99ae0f144006529e" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c381226d6a4cdb995fa379ef50e8b2"
I20260808 04:58:22.933522 6414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8e61a6af89467c99ae0f144006529e" candidate_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911b250ba07645e18eaeae4467fb2df8"
I20260808 04:58:22.933568 6213 raft_consensus.cc:3037] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.933593 6414 raft_consensus.cc:3037] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:22.934176 6213 raft_consensus.cc:2454] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 1.
I20260808 04:58:22.934310 5869 leader_election.cc:302] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c381226d6a4cdb995fa379ef50e8b2, 7970cadb052f4f9c94afd8ca1fbc0af4; no voters:
I20260808 04:58:22.934386 6545 raft_consensus.cc:2781] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:22.934423 6545 raft_consensus.cc:686] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 LEADER]: Becoming Leader. State: Replica: 7970cadb052f4f9c94afd8ca1fbc0af4, State: Running, Role: LEADER
I20260808 04:58:22.934465 6545 consensus_queue.cc:237] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } }
I20260808 04:58:22.934504 6414 raft_consensus.cc:2454] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7970cadb052f4f9c94afd8ca1fbc0af4 in term 1.
I20260808 04:58:22.934664 6545 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: TxnStatusTablet state changed. Reason: New leader 7970cadb052f4f9c94afd8ca1fbc0af4. Latest consensus state: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.934737 6545 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:58:22.935083 5716 catalog_manager.cc:5495] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7970cadb052f4f9c94afd8ca1fbc0af4 (127.3.174.129). New cstate: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } health_report { overall_health: UNKNOWN } } }
I20260808 04:58:22.935207 6590 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 04:58:22.935303 6590 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 04:58:22.943590 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.941151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43698 (local address 127.3.174.129:39985)
0808 04:58:22.941200 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.941202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.941210 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:22.941273 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.941274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.941282 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.941350 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:22.941478 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.941940 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.942996 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.943169 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.943210 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.943405 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:22.943430 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:22.943461 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.943523 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.943540 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 04:58:22.943650 6583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.941057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39985 (local address 127.0.0.1:43698)
0808 04:58:22.941145 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.941157 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.941203 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.941362 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.941363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.941365 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:22.941415 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.941417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.942011 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.942013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.942917 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.942920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.943224 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.943225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.943302 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.943323 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:22.943464 (+ 141us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:22.943470 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.943523 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.943571 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 04:58:22.944660 6414 raft_consensus.cc:1261] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Refusing update from remote peer 7970cadb052f4f9c94afd8ca1fbc0af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:22.944679 6213 raft_consensus.cc:1261] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Refusing update from remote peer 7970cadb052f4f9c94afd8ca1fbc0af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:22.944890 6545 consensus_queue.cc:1035] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:22.945106 6557 consensus_queue.cc:1035] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:22.945648 6541 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: TxnStatusTablet state changed. Reason: New leader 7970cadb052f4f9c94afd8ca1fbc0af4. Latest consensus state: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.945729 6541 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:22.945858 6542 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: TxnStatusTablet state changed. Reason: New leader 7970cadb052f4f9c94afd8ca1fbc0af4. Latest consensus state: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.945933 6542 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:22.946417 6557 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.946523 6557 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:58:22.946653 6549 mvcc.cc:205] Tried to move back new op lower bound from 7316132261660028928 to 7316132261619929088. Current Snapshot: MvccSnapshot[applied={T|T < 7316132261660028928}]
I20260808 04:58:22.946852 6538 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.946923 6538 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 7970cadb052f4f9c94afd8ca1fbc0af4: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:58:22.947207 6541 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.947291 6541 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 04c381226d6a4cdb995fa379ef50e8b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:22.947288 6542 tablet_replica.cc:439] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7970cadb052f4f9c94afd8ca1fbc0af4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39985 } } peers { permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 } } peers { permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 } } }
I20260808 04:58:22.947373 6542 tablet_replica.cc:442] T 7d8e61a6af89467c99ae0f144006529e P 911b250ba07645e18eaeae4467fb2df8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:22.953913 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.951340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:59033 (local address 127.3.174.129:39985)
0808 04:58:22.951375 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.951377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.951743 (+ 366us) server_negotiation.cc:407] Connection header received
0808 04:58:22.951766 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.951767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.951777 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.951826 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:22.951986 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.952426 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.953207 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.953470 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.953528 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.953681 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:22.953713 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:22.953763 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.953815 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.953833 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":15}
I20260808 04:58:22.953964 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.951298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39985 (local address 127.3.174.129:59033)
0808 04:58:22.951662 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:22.951671 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:22.951728 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:22.951829 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:22.951830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:22.951831 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:22.951930 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.951932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.952488 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.952489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.953133 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:22.953136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.953535 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.953536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:22.953601 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.953615 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:22.953773 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:22.953775 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:58:22.953815 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 04:58:22.953854 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":253,"threads_started":1}
I20260808 04:58:22.956136 6213 raft_consensus.cc:1261] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Refusing update from remote peer 7970cadb052f4f9c94afd8ca1fbc0af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:22.956136 6414 raft_consensus.cc:1261] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Refusing update from remote peer 7970cadb052f4f9c94afd8ca1fbc0af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:22.956318 6545 consensus_queue.cc:1035] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c381226d6a4cdb995fa379ef50e8b2" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:22.956406 6538 consensus_queue.cc:1035] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "911b250ba07645e18eaeae4467fb2df8" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:22.971827 5716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:22.971916 5716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:22.972507 5716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa86c3b782924fa09bc452fb3c49dc5d: Sending DeleteTablet for 3 replicas of tablet 84c340885cdd4d9d8fbd3022053e7474
I20260808 04:58:22.972764 6180 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c340885cdd4d9d8fbd3022053e7474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:22 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:56274
I20260808 04:58:22.972868 6564 tablet_replica.cc:330] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: stopping tablet replica
I20260808 04:58:22.972877 6385 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c340885cdd4d9d8fbd3022053e7474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:22 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:56282
I20260808 04:58:22.972939 6564 raft_consensus.cc:2229] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:22.972971 6564 raft_consensus.cc:2258] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:22.973042 6566 tablet_replica.cc:330] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: stopping tablet replica
I20260808 04:58:22.973094 6566 raft_consensus.cc:2229] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:22.973126 6566 raft_consensus.cc:2258] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:22.973416 5946 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c340885cdd4d9d8fbd3022053e7474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:22 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:43668
I20260808 04:58:22.973510 6568 tablet_replica.cc:330] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: stopping tablet replica
I20260808 04:58:22.973553 6568 raft_consensus.cc:2229] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:22.973580 6564 ts_tablet_manager.cc:1905] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:22.973604 6568 raft_consensus.cc:2258] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:22.973852 6568 ts_tablet_manager.cc:1905] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:22.975420 6564 ts_tablet_manager.cc:1918] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:22.975432 6568 ts_tablet_manager.cc:1918] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:22.975487 6568 log.cc:1196] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/84c340885cdd4d9d8fbd3022053e7474
I20260808 04:58:22.975487 6564 log.cc:1196] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/84c340885cdd4d9d8fbd3022053e7474
I20260808 04:58:22.975514 6566 ts_tablet_manager.cc:1905] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:22.975710 6564 ts_tablet_manager.cc:1939] T 84c340885cdd4d9d8fbd3022053e7474 P 04c381226d6a4cdb995fa379ef50e8b2: Deleting consensus metadata
I20260808 04:58:22.975734 6568 ts_tablet_manager.cc:1939] T 84c340885cdd4d9d8fbd3022053e7474 P 7970cadb052f4f9c94afd8ca1fbc0af4: Deleting consensus metadata
I20260808 04:58:22.976042 5693 catalog_manager.cc:4841] TS 7970cadb052f4f9c94afd8ca1fbc0af4 (127.3.174.129:39985): tablet 84c340885cdd4d9d8fbd3022053e7474 (table test-table [id=dc8f5f86a0fd4571abeeb694129b24af]) successfully deleted
I20260808 04:58:22.976056 5692 catalog_manager.cc:4841] TS 04c381226d6a4cdb995fa379ef50e8b2 (127.3.174.130:36085): tablet 84c340885cdd4d9d8fbd3022053e7474 (table test-table [id=dc8f5f86a0fd4571abeeb694129b24af]) successfully deleted
I20260808 04:58:22.977165 6566 ts_tablet_manager.cc:1918] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:22.977238 6566 log.cc:1196] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/84c340885cdd4d9d8fbd3022053e7474
I20260808 04:58:22.977458 6566 ts_tablet_manager.cc:1939] T 84c340885cdd4d9d8fbd3022053e7474 P 911b250ba07645e18eaeae4467fb2df8: Deleting consensus metadata
I20260808 04:58:22.977782 5693 catalog_manager.cc:4841] TS 911b250ba07645e18eaeae4467fb2df8 (127.3.174.131:40791): tablet 84c340885cdd4d9d8fbd3022053e7474 (table test-table [id=dc8f5f86a0fd4571abeeb694129b24af]) successfully deleted
Aug 08 04:58:22 dist-test-slave-gjw7 krb5kdc[5643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165102, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:22.985669 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.975696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43706 (local address 127.3.174.129:39985)
0808 04:58:22.975738 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.975740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.976396 (+ 656us) server_negotiation.cc:407] Connection header received
0808 04:58:22.976443 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.976444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.976454 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.976508 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.977344 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.978007 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.978751 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.979010 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.979077 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.982478 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.982487 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.982488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.982496 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.984448 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.984801 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.984805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.984805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.984882 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.985054 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.985054 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.985055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.985219 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.985382 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.985451 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.985530 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 04:58:22.986001 6043 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43706
I20260808 04:58:22.993247 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:22.987002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45538 (local address 127.3.174.190:40419)
0808 04:58:22.987049 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:22.987051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:22.987110 (+ 59us) server_negotiation.cc:407] Connection header received
0808 04:58:22.987183 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:22.987184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:22.987195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:22.987242 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:22.987993 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.988518 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.989308 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:22.989510 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:22.989574 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:22.990841 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:22.990843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:22.990843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:22.990850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:22.992159 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.992558 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.992559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.992560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.992616 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:22.992795 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:22.992796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:22.992797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:22.992907 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:22.992992 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:22.993085 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 04:58:22.993158 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 04:58:22.993553 5716 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45538
I20260808 04:58:22.994233 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 5846
I20260808 04:58:23.005789 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 6071
I20260808 04:58:23.015074 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 6298
I20260808 04:58:23.028998 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 5666
2026-08-08T04:58:23Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2224 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:23 dist-test-slave-gjw7 krb5kdc[6612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:23 dist-test-slave-gjw7 krb5kdc[6612](info): set up 2 sockets
Aug 08 04:58:23 dist-test-slave-gjw7 krb5kdc[6612](info): commencing operation
krb5kdc: starting...
W20260808 04:58:24.206300 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:24.322031 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:37885
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:42939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:37885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:24.404561 6640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:24.404755 6640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:24.404776 6640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:24.405997 6640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:24.406039 6640 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 04:58:24.406051 6640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:24.406062 6640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:24.406073 6640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:24.406083 6640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:24.407521 6640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:37885
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:37885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:24.407727 6640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:24.407895 6640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:24.412571 6654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:24.412703 6640 server_base.cc:1034] running on GCE node
W20260808 04:58:24.412576 6650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:24.412580 6651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:24.413102 6640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:24.413429 6640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:24.414577 6640 hybrid_clock.cc:648] HybridClock initialized: now 1786165104414563 us; error 36 us; skew 500 ppm
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:24.417038 6640 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:24.417511 6640 webserver.cc:460] Webserver started at http://127.3.174.190:36955/ using document root <none> and password file <none>
I20260808 04:58:24.417723 6640 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:24.417780 6640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:24.417908 6640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:24.418958 6640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "dd7d192ecb8344838a0ab3a9e6575353"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "07a7d9c3a5cee7fff46965bc59a65dce"
server_key_iv: "fa05a82ed654269cdd1b94d86a69643b"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.419298 6640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "dd7d192ecb8344838a0ab3a9e6575353"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "07a7d9c3a5cee7fff46965bc59a65dce"
server_key_iv: "fa05a82ed654269cdd1b94d86a69643b"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.420611 6640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 04:58:24.421334 6665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:24.421484 6640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:24.421571 6640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "dd7d192ecb8344838a0ab3a9e6575353"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "07a7d9c3a5cee7fff46965bc59a65dce"
server_key_iv: "fa05a82ed654269cdd1b94d86a69643b"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.421638 6640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:24.432531 6640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:24.434360 6640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:24.434485 6640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:24.438968 6640 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:37885
I20260808 04:58:24.439024 6758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:37885 every 8 connection(s)
I20260808 04:58:24.439321 6640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:24.439937 6759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:24.442239 6759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353: Bootstrap starting.
I20260808 04:58:24.442790 6759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:24.443048 6759 log.cc:824] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:24.443668 6759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353: No bootstrap required, opened a new log
I20260808 04:58:24.446247 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 6640
I20260808 04:58:24.446336 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:24.446481 3770 external_mini_cluster.cc:1428] Setting key 2d8df3e98fe4cdd5de434f96738c77e4
I20260808 04:58:24.447158 6759 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7d192ecb8344838a0ab3a9e6575353" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37885 } }
I20260808 04:58:24.447340 6759 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:24.447376 6759 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7d192ecb8344838a0ab3a9e6575353, State: Initialized, Role: FOLLOWER
I20260808 04:58:24.447510 6759 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7d192ecb8344838a0ab3a9e6575353" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37885 } }
I20260808 04:58:24.447594 6759 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:24.447662 6759 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:24.447726 6759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:24.448596 6759 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7d192ecb8344838a0ab3a9e6575353" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37885 } }
I20260808 04:58:24.448716 6759 leader_election.cc:302] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd7d192ecb8344838a0ab3a9e6575353; no voters:
I20260808 04:58:24.448879 6759 leader_election.cc:288] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:24.448912 6765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:24.449088 6765 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [term 1 LEADER]: Becoming Leader. State: Replica: dd7d192ecb8344838a0ab3a9e6575353, State: Running, Role: LEADER
I20260808 04:58:24.449116 6759 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:24.449183 6765 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7d192ecb8344838a0ab3a9e6575353" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37885 } }
I20260808 04:58:24.449635 6768 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd7d192ecb8344838a0ab3a9e6575353. Latest consensus state: current_term: 1 leader_uuid: "dd7d192ecb8344838a0ab3a9e6575353" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7d192ecb8344838a0ab3a9e6575353" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37885 } } }
I20260808 04:58:24.449718 6768 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:24.450290 6779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:24.450187 6766 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd7d192ecb8344838a0ab3a9e6575353" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7d192ecb8344838a0ab3a9e6575353" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37885 } } }
I20260808 04:58:24.450518 6766 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:24.450852 6779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:24.452523 6779 catalog_manager.cc:1347] Generated new cluster ID: 9c353cfed2594e9189e8426b38806b15
I20260808 04:58:24.452575 6779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:24.456892 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.447018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53100 (local address 127.3.174.190:37885)
0808 04:58:24.447243 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:24.447253 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:24.447286 (+ 33us) server_negotiation.cc:407] Connection header received
0808 04:58:24.447415 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:24.447422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:24.447479 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:24.447616 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:24.448289 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.449644 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.450677 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.451319 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.451423 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.453828 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:24.453833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:24.453833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:24.453846 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:24.455359 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.455669 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.455672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.455672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.455784 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.455967 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.455968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.455968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.456284 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:24.456434 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:24.456541 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 04:58:24.456673 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":112,"threads_started":1}
I20260808 04:58:24.467269 6779 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:24.467751 6779 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:24.480137 6779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353: Generated new TSK 0
I20260808 04:58:24.480324 6779 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:24.510032 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37885
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:42939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:24.591955 6806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:24.592149 6806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:24.592170 6806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:24.593271 6806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:24.593334 6806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:24.593364 6806 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 04:58:24.593387 6806 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:58:24.593410 6806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:24.594960 6806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.174.190:37885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:24.595185 6806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:24.595405 6806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:24.600059 6819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:24.600067 6825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:24.600067 6816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:24.600294 6806 server_base.cc:1034] running on GCE node
I20260808 04:58:24.600482 6806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:24.600684 6806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:24.601815 6806 hybrid_clock.cc:648] HybridClock initialized: now 1786165104601801 us; error 27 us; skew 500 ppm
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:24.604218 6806 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:24.604673 6806 webserver.cc:460] Webserver started at http://127.3.174.129:36861/ using document root <none> and password file <none>
I20260808 04:58:24.604858 6806 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:24.604897 6806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:24.604990 6806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:24.605971 6806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "17a0b77ad1554b2a84cea040d95dc482"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "7ff598dc8b0f2aa8ff8d384ddd4ef587"
server_key_iv: "5c6a8d5333e8d8a660718867e63c5eae"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.606268 6806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "17a0b77ad1554b2a84cea040d95dc482"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "7ff598dc8b0f2aa8ff8d384ddd4ef587"
server_key_iv: "5c6a8d5333e8d8a660718867e63c5eae"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.607465 6806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:24.608136 6849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:24.608309 6806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:24.608389 6806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "17a0b77ad1554b2a84cea040d95dc482"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "7ff598dc8b0f2aa8ff8d384ddd4ef587"
server_key_iv: "5c6a8d5333e8d8a660718867e63c5eae"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.608451 6806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:24.622632 6806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:24.625130 6806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:24.625273 6806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:24.625816 6806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:24.625859 6806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:24.625890 6806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:24.625907 6806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:24.632942 6806 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:40815
I20260808 04:58:24.633023 7050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:40815 every 8 connection(s)
I20260808 04:58:24.633379 6806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:58:24.634815 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 6806
I20260808 04:58:24.634904 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:24.635097 3770 external_mini_cluster.cc:1428] Setting key 55dfb2f6a1250082d5a71267f764dfad
I20260808 04:58:24.635633 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.626261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:46081 (local address 127.3.174.190:37885)
0808 04:58:24.626327 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:24.626329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:24.626568 (+ 239us) server_negotiation.cc:407] Connection header received
0808 04:58:24.627307 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:24.627308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:24.627325 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:24.627378 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:24.628234 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.628997 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.630473 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.630736 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.630801 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.632852 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:24.632854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:24.632854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:24.632862 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:24.634294 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.634633 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.634634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.634634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.634694 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.634904 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.634905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.634905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.635040 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:24.635108 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:24.635469 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 04:58:24.635549 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 04:58:24.635665 6870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.626178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37885 (local address 127.3.174.129:46081)
0808 04:58:24.626371 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:24.626422 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:24.627183 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:24.627472 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:24.627478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:24.627837 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:24.628131 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.628138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.629090 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.629091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.630388 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.630391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.630828 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.630829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.630954 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.631372 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:24.631379 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:24.632766 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:24.634358 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.634365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.634367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.634578 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.634764 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.634764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.634765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.634848 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.635107 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:24.635114 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:24.635268 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 04:58:24.635379 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 04:58:24.635512 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260808 04:58:24.637547 7051 heartbeater.cc:347] Connected to a master server at 127.3.174.190:37885
I20260808 04:58:24.637672 7051 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:24.637887 7051 heartbeater.cc:511] Master 127.3.174.190:37885 requested a full tablet report, sending...
I20260808 04:58:24.638393 6691 ts_manager.cc:194] Registered new tserver with Master: 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815)
I20260808 04:58:24.639002 6691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:46081
I20260808 04:58:24.642506 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.637257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:33003 (local address 127.3.174.190:37885)
0808 04:58:24.637320 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:24.637322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:24.637380 (+ 58us) server_negotiation.cc:407] Connection header received
0808 04:58:24.637441 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:24.637442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:24.637451 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:24.637503 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:24.637840 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.638489 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.639266 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.639498 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.639556 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.640308 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:24.640309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:24.640310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:24.640316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:24.641556 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.641846 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.641848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.641848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.641896 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.642052 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.642053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.642053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.642153 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:24.642205 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:24.642352 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 04:58:24.642417 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 04:58:24.642524 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.637164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37885 (local address 127.3.174.129:33003)
0808 04:58:24.637322 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:24.637333 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:24.637394 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:24.637517 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:24.637517 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:24.637700 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:24.637761 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.637763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.638599 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.638600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.639210 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.639212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.639563 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.639564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.639639 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.639975 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:24.639978 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:24.640234 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:24.641632 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.641633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.641634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.641790 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.641947 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.641947 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.641948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.642005 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.642205 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:24.642205 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:24.642253 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 04:58:24.642323 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 04:58:24.642417 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:24.661394 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37885
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:42939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:24.744022 7061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:24.744234 7061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:24.744269 7061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:24.745412 7061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:24.745450 7061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:24.745476 7061 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 04:58:24.745491 7061 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:58:24.745514 7061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:24.746979 7061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.174.190:37885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:24.747210 7061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:24.747418 7061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:24.752415 7069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:24.752424 7071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:24.752492 7061 server_base.cc:1034] running on GCE node
W20260808 04:58:24.752457 7068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:24.752785 7061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:24.753026 7061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:24.754179 7061 hybrid_clock.cc:648] HybridClock initialized: now 1786165104754163 us; error 29 us; skew 500 ppm
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:24.756657 7061 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:24.757067 7061 webserver.cc:460] Webserver started at http://127.3.174.130:43993/ using document root <none> and password file <none>
I20260808 04:58:24.757265 7061 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:24.757337 7061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:24.757467 7061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:24.758483 7061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "41afda761d84417683ca44c20c269049"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "f706b05c2ec0a884e8c6b3c98cb9a680"
server_key_iv: "18228c0d414c7cfc06fb1d16e0c7672d"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.758816 7061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "41afda761d84417683ca44c20c269049"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "f706b05c2ec0a884e8c6b3c98cb9a680"
server_key_iv: "18228c0d414c7cfc06fb1d16e0c7672d"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.760042 7061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:24.760668 7089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:24.760809 7061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:24.760895 7061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "41afda761d84417683ca44c20c269049"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "f706b05c2ec0a884e8c6b3c98cb9a680"
server_key_iv: "18228c0d414c7cfc06fb1d16e0c7672d"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.760957 7061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:24.797365 7061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:24.799486 7061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:24.799630 7061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:24.800217 7061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:24.800266 7061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:24.800302 7061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:24.800318 7061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:24.807140 7061 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:41423
I20260808 04:58:24.807535 7061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:58:24.807585 7301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:41423 every 8 connection(s)
I20260808 04:58:24.810042 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.800753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:59031 (local address 127.3.174.190:37885)
0808 04:58:24.800826 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:24.800829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:24.801013 (+ 184us) server_negotiation.cc:407] Connection header received
0808 04:58:24.801616 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:24.801618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:24.801631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:24.801689 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:24.802333 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.803095 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.804631 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.804955 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.805018 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.807541 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:24.807542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:24.807543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:24.807556 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:24.808830 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.809267 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.809268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.809268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.809357 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.809562 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.809562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.809562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.809666 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:24.809714 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:24.809916 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 04:58:24.809977 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 04:58:24.810230 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.800710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37885 (local address 127.3.174.130:59031)
0808 04:58:24.800880 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:24.800925 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:24.801513 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:24.801763 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:24.801768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:24.802044 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:24.802242 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.802248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.803234 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.803236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.804515 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.804519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.805070 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.805071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.805236 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.805731 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:24.805743 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:24.807442 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:24.808905 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.808911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.808913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.809142 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.809408 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.809409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.809409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.809513 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.809710 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:24.809713 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:24.809824 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 04:58:24.809901 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 04:58:24.810019 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260808 04:58:24.810724 7302 heartbeater.cc:347] Connected to a master server at 127.3.174.190:37885
I20260808 04:58:24.810859 7302 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:24.811120 7302 heartbeater.cc:511] Master 127.3.174.190:37885 requested a full tablet report, sending...
I20260808 04:58:24.811584 6692 ts_manager.cc:194] Registered new tserver with Master: 41afda761d84417683ca44c20c269049 (127.3.174.130:41423)
I20260808 04:58:24.812068 6692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:59031
I20260808 04:58:24.816622 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.811737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37885 (local address 127.3.174.130:34461)
0808 04:58:24.811890 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:24.811899 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:24.811953 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:24.812077 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:24.812077 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:24.812262 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:24.812309 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.812311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.812936 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.812937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.813478 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.813480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.813791 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.813792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.813841 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.814142 (+ 301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:24.814146 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:24.814401 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:24.815849 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.815850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.815850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.815994 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.816129 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.816130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.816130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.816173 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.816416 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:24.816417 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:24.816448 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 04:58:24.816492 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 04:58:24.816559 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":105,"threads_started":1}
I20260808 04:58:24.816666 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.811784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:34461 (local address 127.3.174.190:37885)
0808 04:58:24.811825 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:24.811827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:24.811957 (+ 130us) server_negotiation.cc:407] Connection header received
0808 04:58:24.812001 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:24.812002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:24.812012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:24.812072 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:24.812374 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.812850 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.813554 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.813742 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.813790 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.814494 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:24.814495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:24.814495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:24.814502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:24.815791 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.816046 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.816047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.816047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.816090 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.816238 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.816238 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.816239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.816372 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:24.816420 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:24.816513 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 04:58:24.816585 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 04:58:24.816700 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 7061
I20260808 04:58:24.816777 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:24.816951 3770 external_mini_cluster.cc:1428] Setting key dd2c9a7604ea82aec2ec99e3a6938caa
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:24.842476 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37885
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:42939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:24.926740 7318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:24.926945 7318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:24.926968 7318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:24.928073 7318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:24.928109 7318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:24.928128 7318 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 04:58:24.928138 7318 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:58:24.928153 7318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:24.929637 7318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.174.190:37885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:24.929838 7318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:24.930016 7318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:24.934911 7330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:24.935019 7327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:24.934943 7333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:24.935117 7318 server_base.cc:1034] running on GCE node
I20260808 04:58:24.935521 7318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:24.935810 7318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:24.936949 7318 hybrid_clock.cc:648] HybridClock initialized: now 1786165104936941 us; error 34 us; skew 500 ppm
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:24.939378 7318 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:24.939914 7318 webserver.cc:460] Webserver started at http://127.3.174.131:41001/ using document root <none> and password file <none>
I20260808 04:58:24.940172 7318 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:24.940228 7318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:24.940392 7318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:24.941515 7318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "b88c388178f74cacb3b7b5cb65039610"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "fcea62b02ce51616d098aac4a7a8f91d"
server_key_iv: "66dcc540ffa8c6579e00b9290151c693"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.941859 7318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "b88c388178f74cacb3b7b5cb65039610"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "fcea62b02ce51616d098aac4a7a8f91d"
server_key_iv: "66dcc540ffa8c6579e00b9290151c693"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.943111 7318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:24.943869 7344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:24.944018 7318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:24.944105 7318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "b88c388178f74cacb3b7b5cb65039610"
format_stamp: "Formatted at 2026-08-08 04:58:24 on dist-test-slave-gjw7"
server_key: "fcea62b02ce51616d098aac4a7a8f91d"
server_key_iv: "66dcc540ffa8c6579e00b9290151c693"
server_key_version: "encryptionkey@0"
I20260808 04:58:24.944167 7318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:24.959157 7318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:24.960999 7318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:24.961130 7318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:24.961834 7318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:24.961884 7318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:24.961915 7318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:24.961951 7318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:24.969354 7318 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:33841
I20260808 04:58:24.969693 7318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 04:58:24.970064 7549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:33841 every 8 connection(s)
I20260808 04:58:24.971630 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.962137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:36537 (local address 127.3.174.190:37885)
0808 04:58:24.962192 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:24.962195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:24.962446 (+ 251us) server_negotiation.cc:407] Connection header received
0808 04:58:24.963180 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:24.963181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:24.963195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:24.963283 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:24.964045 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.964565 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.965665 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.965855 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.965911 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.968816 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:24.968817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:24.968817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:24.968826 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:24.970321 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.970741 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.970743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.970743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.970811 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.971024 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.971025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.971026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.971142 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:24.971198 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:24.971461 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 04:58:24.971542 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 04:58:24.971786 7362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.962147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37885 (local address 127.3.174.131:36537)
0808 04:58:24.962318 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:24.962362 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:24.963049 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:24.963365 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:24.963370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:24.963747 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:24.963955 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.963960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.964665 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.964666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.965587 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.965590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.965923 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.965924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.966047 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.966541 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:24.966552 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:24.968725 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:24.970392 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.970407 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.970410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.970675 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.970862 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.970863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.970863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.970969 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.971195 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:24.971201 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:24.971339 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 04:58:24.971429 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:58:24.971580 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260808 04:58:24.972236 7550 heartbeater.cc:347] Connected to a master server at 127.3.174.190:37885
I20260808 04:58:24.972324 7550 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:24.972498 7550 heartbeater.cc:511] Master 127.3.174.190:37885 requested a full tablet report, sending...
I20260808 04:58:24.972872 6691 ts_manager.cc:194] Registered new tserver with Master: b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841)
I20260808 04:58:24.973344 6691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:36537
I20260808 04:58:24.978008 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.973348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:60809 (local address 127.3.174.190:37885)
0808 04:58:24.973391 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:24.973393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:24.973515 (+ 122us) server_negotiation.cc:407] Connection header received
0808 04:58:24.973559 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:24.973560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:24.973570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:24.973634 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:24.973981 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.974423 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.975039 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.975204 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.975282 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.976089 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:24.976090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:24.976090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:24.976094 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:24.977128 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.977472 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.977472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.977473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.977521 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:24.977637 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:24.977637 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:24.977638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:24.977720 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:24.977773 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:24.977899 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 04:58:24.977951 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 04:58:24.978047 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 7318
I20260808 04:58:24.978071 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.973269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37885 (local address 127.3.174.131:60809)
0808 04:58:24.973447 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:24.973457 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:24.973510 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:24.973648 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:24.973648 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:24.973847 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:24.973910 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.973912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.974493 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.974494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.974982 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:24.974984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.975249 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.975249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:24.975311 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:24.975677 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:24.975682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:24.976028 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:24.977189 (+ 1161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.977190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.977191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.977408 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.977557 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:24.977557 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:24.977558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:24.977605 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:24.977768 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:24.977768 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:24.977800 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 04:58:24.977878 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 04:58:24.977975 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260808 04:58:24.978122 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:24.978312 3770 external_mini_cluster.cc:1428] Setting key d6c0489a06cf3c3cfab280ee8d82d337
I20260808 04:58:24.979103 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:58:24 dist-test-slave-gjw7 krb5kdc[6612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:25.002454 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:24.996012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.3.174.190:37885)
0808 04:58:24.996064 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:24.996066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:24.996075 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:24.996139 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:24.996140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:24.996150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:24.996195 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:24.996792 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.997272 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.998007 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:24.998190 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:24.998245 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.000431 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:25.000433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:25.000433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:25.000440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:25.001557 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:25.001892 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:25.001892 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:25.001893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:25.001938 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:25.002081 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:25.002082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:25.002082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:25.002175 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:25.002247 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.002323 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.002386 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 04:58:25.003953 6691 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:25.005560 6691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:25.012544 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.007891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:33841 (local address 127.0.0.1:41690)
0808 04:58:25.008062 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.008111 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.008184 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.008696 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.008699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.008701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:25.008788 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.008790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.010134 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.010135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.011235 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.011236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.012218 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.012218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.012311 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.012327 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.012406 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.012462 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":8,"thread_start_us":45,"threads_started":1}
I20260808 04:58:25.012558 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.008266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41690 (local address 127.3.174.131:33841)
0808 04:58:25.008439 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.008443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.008452 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:25.008494 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.008496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.008513 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.008597 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:25.008870 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.010037 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.011311 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.012116 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.012215 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.012315 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.012424 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.012464 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260808 04:58:25.012584 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.008039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.3.174.130:41423)
0808 04:58:25.008238 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.008243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.008474 (+ 231us) server_negotiation.cc:407] Connection header received
0808 04:58:25.008542 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.008544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.008587 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.008654 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:25.008886 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.009771 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.011311 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.012116 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.012186 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.012252 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.012452 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.012493 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":84,"threads_started":1}
I20260808 04:58:25.013188 7591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.008079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:40815 (local address 127.0.0.1:50524)
0808 04:58:25.008428 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.008437 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.008480 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.008733 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.008734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.008736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:25.008785 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.008787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.011319 (+ 2532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.011320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.012103 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.012106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.012975 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.012976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.013033 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.013038 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.013091 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.013126 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":73,"threads_started":1}
I20260808 04:58:25.013237 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.008218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50524 (local address 127.3.174.129:40815)
0808 04:58:25.008383 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.008387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.008537 (+ 150us) server_negotiation.cc:407] Connection header received
0808 04:58:25.008584 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.008586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.008601 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.008864 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:25.008926 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.010036 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.012179 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.012885 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.012953 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.013006 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.013108 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.013146 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260808 04:58:25.012791 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.007904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:41423 (local address 127.0.0.1:50804)
0808 04:58:25.008399 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.008410 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.008476 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.008719 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.008720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.008723 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:25.008791 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.008794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.009857 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.009858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.011205 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.011217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.012291 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.012292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.012362 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.012366 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.012678 (+ 312us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.012744 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":83,"threads_started":1}
I20260808 04:58:25.013778 7434 tablet_service.cc:1467] Processing CreateTablet for tablet af89e44ee0c144b39ea8b079758946ff (DEFAULT_TABLE table=test-table [id=1e08edeb413e46f0a093db9707d23612]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:25.014165 7434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af89e44ee0c144b39ea8b079758946ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:25.014173 6940 tablet_service.cc:1467] Processing CreateTablet for tablet af89e44ee0c144b39ea8b079758946ff (DEFAULT_TABLE table=test-table [id=1e08edeb413e46f0a093db9707d23612]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:25.014175 7186 tablet_service.cc:1467] Processing CreateTablet for tablet af89e44ee0c144b39ea8b079758946ff (DEFAULT_TABLE table=test-table [id=1e08edeb413e46f0a093db9707d23612]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:25.014526 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af89e44ee0c144b39ea8b079758946ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:25.014526 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af89e44ee0c144b39ea8b079758946ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:25.016739 7606 tablet_bootstrap.cc:492] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: Bootstrap starting.
I20260808 04:58:25.017649 7606 tablet_bootstrap.cc:654] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:25.017982 7606 log.cc:824] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:25.018218 7610 tablet_bootstrap.cc:492] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: Bootstrap starting.
I20260808 04:58:25.018810 7610 tablet_bootstrap.cc:654] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:25.018834 7606 tablet_bootstrap.cc:492] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: No bootstrap required, opened a new log
I20260808 04:58:25.018893 7606 ts_tablet_manager.cc:1397] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 04:58:25.019172 7610 log.cc:824] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:25.019505 7609 tablet_bootstrap.cc:492] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: Bootstrap starting.
I20260808 04:58:25.019893 7610 tablet_bootstrap.cc:492] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: No bootstrap required, opened a new log
I20260808 04:58:25.019965 7610 ts_tablet_manager.cc:1397] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:25.020205 7609 tablet_bootstrap.cc:654] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:25.020530 7609 log.cc:824] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:25.020972 7606 raft_consensus.cc:348] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } }
I20260808 04:58:25.021150 7606 raft_consensus.cc:374] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:25.021225 7606 raft_consensus.cc:729] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88c388178f74cacb3b7b5cb65039610, State: Initialized, Role: FOLLOWER
I20260808 04:58:25.021410 7609 tablet_bootstrap.cc:492] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: No bootstrap required, opened a new log
I20260808 04:58:25.021394 7606 consensus_queue.cc:260] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } }
I20260808 04:58:25.021473 7609 ts_tablet_manager.cc:1397] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:25.021677 7606 ts_tablet_manager.cc:1428] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 04:58:25.021716 7550 heartbeater.cc:503] Master 127.3.174.190:37885 was elected leader, sending a full tablet report...
I20260808 04:58:25.022154 7610 raft_consensus.cc:348] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } }
I20260808 04:58:25.022296 7610 raft_consensus.cc:374] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:25.022327 7610 raft_consensus.cc:729] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41afda761d84417683ca44c20c269049, State: Initialized, Role: FOLLOWER
I20260808 04:58:25.022432 7610 consensus_queue.cc:260] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } }
I20260808 04:58:25.022836 7610 ts_tablet_manager.cc:1428] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 04:58:25.022871 7302 heartbeater.cc:503] Master 127.3.174.190:37885 was elected leader, sending a full tablet report...
I20260808 04:58:25.022995 7609 raft_consensus.cc:348] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } }
I20260808 04:58:25.023134 7609 raft_consensus.cc:374] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:25.023166 7609 raft_consensus.cc:729] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a0b77ad1554b2a84cea040d95dc482, State: Initialized, Role: FOLLOWER
I20260808 04:58:25.023257 7609 consensus_queue.cc:260] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } }
I20260808 04:58:25.023473 7609 ts_tablet_manager.cc:1428] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 04:58:25.023517 7051 heartbeater.cc:503] Master 127.3.174.190:37885 was elected leader, sending a full tablet report...
W20260808 04:58:25.058326 7303 tablet.cc:2369] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:25.074407 7619 raft_consensus.cc:482] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:25.074566 7619 raft_consensus.cc:504] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } }
I20260808 04:58:25.074975 7619 leader_election.cc:288] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841), 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815)
I20260808 04:58:25.078111 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.075282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:55591 (local address 127.3.174.131:33841)
0808 04:58:25.075330 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.075332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.075376 (+ 44us) server_negotiation.cc:407] Connection header received
0808 04:58:25.075407 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.075408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.075418 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.075457 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:25.075635 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.076228 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.077327 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.077806 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.077864 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.077873 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.078034 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.078059 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 04:58:25.078145 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.075193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:33841 (local address 127.3.174.130:55591)
0808 04:58:25.075267 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.075280 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.075328 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.075487 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.075488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.075490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:25.075564 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.075565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.076348 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.076349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.077211 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.077213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.077877 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.077878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.077964 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.077969 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.078027 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.078069 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 04:58:25.078521 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af89e44ee0c144b39ea8b079758946ff" candidate_uuid: "41afda761d84417683ca44c20c269049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b88c388178f74cacb3b7b5cb65039610" is_pre_election: true
I20260808 04:58:25.078711 7466 raft_consensus.cc:2454] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41afda761d84417683ca44c20c269049 in term 0.
I20260808 04:58:25.078799 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.075310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:41321 (local address 127.3.174.129:40815)
0808 04:58:25.075349 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.075350 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.075645 (+ 295us) server_negotiation.cc:407] Connection header received
0808 04:58:25.075676 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.075677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.075686 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.075723 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:25.075925 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.076642 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.077638 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.078150 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.078282 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.078291 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.078690 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.078736 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 04:58:25.078876 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.075307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:40815 (local address 127.3.174.130:41321)
0808 04:58:25.075565 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.075574 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.075632 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.075742 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.075743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.075745 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:25.075817 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.075819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.076730 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.076731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.077553 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.077556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.078426 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.078428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.078603 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.078609 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.078671 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.078707 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":194,"threads_started":1}
I20260808 04:58:25.078976 7091 leader_election.cc:302] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41afda761d84417683ca44c20c269049, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:25.079092 7619 raft_consensus.cc:2781] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:25.079164 7619 raft_consensus.cc:482] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:25.079182 7619 raft_consensus.cc:3037] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:25.079334 6969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af89e44ee0c144b39ea8b079758946ff" candidate_uuid: "41afda761d84417683ca44c20c269049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a0b77ad1554b2a84cea040d95dc482" is_pre_election: true
I20260808 04:58:25.079468 6969 raft_consensus.cc:2454] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41afda761d84417683ca44c20c269049 in term 0.
I20260808 04:58:25.080335 7619 raft_consensus.cc:504] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } }
I20260808 04:58:25.080456 7619 leader_election.cc:288] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [CANDIDATE]: Term 1 election: Requested vote from peers b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841), 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815)
I20260808 04:58:25.080626 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af89e44ee0c144b39ea8b079758946ff" candidate_uuid: "41afda761d84417683ca44c20c269049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b88c388178f74cacb3b7b5cb65039610"
I20260808 04:58:25.080670 6969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af89e44ee0c144b39ea8b079758946ff" candidate_uuid: "41afda761d84417683ca44c20c269049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a0b77ad1554b2a84cea040d95dc482"
I20260808 04:58:25.080718 7466 raft_consensus.cc:3037] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:25.080771 6969 raft_consensus.cc:3037] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:25.081686 6969 raft_consensus.cc:2454] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41afda761d84417683ca44c20c269049 in term 1.
I20260808 04:58:25.081698 7466 raft_consensus.cc:2454] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41afda761d84417683ca44c20c269049 in term 1.
I20260808 04:58:25.081866 7091 leader_election.cc:302] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41afda761d84417683ca44c20c269049, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:25.081983 7619 raft_consensus.cc:2781] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:25.082147 7619 raft_consensus.cc:686] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 1 LEADER]: Becoming Leader. State: Replica: 41afda761d84417683ca44c20c269049, State: Running, Role: LEADER
I20260808 04:58:25.082270 7619 consensus_queue.cc:237] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } }
I20260808 04:58:25.083102 6691 catalog_manager.cc:5495] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41afda761d84417683ca44c20c269049 (127.3.174.130). New cstate: current_term: 1 leader_uuid: "41afda761d84417683ca44c20c269049" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } health_report { overall_health: UNKNOWN } } }
Aug 08 04:58:25 dist-test-slave-gjw7 krb5kdc[6612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165104, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:25.124953 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.118095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37885 (local address 127.0.0.1:53120)
0808 04:58:25.118327 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.118338 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.118422 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.118542 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.118543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.118855 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:25.118914 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.118916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.119554 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.119555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.120100 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.120102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.120439 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.120440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.120476 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.120995 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:25.121000 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:25.122435 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:25.123876 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:25.123883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:25.123886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:25.124110 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:25.124356 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:25.124357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:25.124357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:25.124423 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:25.124633 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:25.124642 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:25.124688 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.124745 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.124858 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
I20260808 04:58:25.125008 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.118181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.3.174.190:37885)
0808 04:58:25.118299 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.118301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.118412 (+ 111us) server_negotiation.cc:407] Connection header received
0808 04:58:25.118468 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.118469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.118479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.118529 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:25.118979 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.119492 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.120174 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.120384 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.120442 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.122531 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:25.122532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:25.122532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:25.122539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:25.123804 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:25.124175 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:25.124176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:25.124176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:25.124283 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:25.124473 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:25.124474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:25.124474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:25.124585 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:25.124629 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.124780 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.124906 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":90}
I20260808 04:58:25.126969 6691 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:53120:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 04:58:25.127152 6691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:25.128582 6940 tablet_service.cc:1467] Processing CreateTablet for tablet bbc54472868b4814a6b2d078131920f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=15f1d9faf6ec421e84b3b90aae16af87]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:58:25.128614 7434 tablet_service.cc:1467] Processing CreateTablet for tablet bbc54472868b4814a6b2d078131920f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=15f1d9faf6ec421e84b3b90aae16af87]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:58:25.128610 7186 tablet_service.cc:1467] Processing CreateTablet for tablet bbc54472868b4814a6b2d078131920f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=15f1d9faf6ec421e84b3b90aae16af87]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:58:25.128763 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbc54472868b4814a6b2d078131920f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:25.128772 7434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbc54472868b4814a6b2d078131920f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:25.128777 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbc54472868b4814a6b2d078131920f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:25.130014 7609 tablet_bootstrap.cc:492] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: Bootstrap starting.
I20260808 04:58:25.130339 7610 tablet_bootstrap.cc:492] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: Bootstrap starting.
I20260808 04:58:25.130754 7609 tablet_bootstrap.cc:654] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:25.130789 7610 tablet_bootstrap.cc:654] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:25.131157 7606 tablet_bootstrap.cc:492] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: Bootstrap starting.
I20260808 04:58:25.131475 7610 tablet_bootstrap.cc:492] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: No bootstrap required, opened a new log
I20260808 04:58:25.131522 7610 ts_tablet_manager.cc:1397] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:25.131683 7606 tablet_bootstrap.cc:654] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:25.131672 7609 tablet_bootstrap.cc:492] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: No bootstrap required, opened a new log
I20260808 04:58:25.131675 7610 raft_consensus.cc:348] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.131731 7610 raft_consensus.cc:374] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:25.131736 7609 ts_tablet_manager.cc:1397] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 04:58:25.131749 7610 raft_consensus.cc:729] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41afda761d84417683ca44c20c269049, State: Initialized, Role: FOLLOWER
I20260808 04:58:25.131795 7610 consensus_queue.cc:260] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.131881 7610 ts_tablet_manager.cc:1428] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:25.131884 7609 raft_consensus.cc:348] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.131960 7609 raft_consensus.cc:374] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:25.131985 7609 raft_consensus.cc:729] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a0b77ad1554b2a84cea040d95dc482, State: Initialized, Role: FOLLOWER
I20260808 04:58:25.131989 7619 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.132059 7619 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:25.132046 7609 consensus_queue.cc:260] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.132208 7621 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.132300 7621 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:25.132467 7609 ts_tablet_manager.cc:1428] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:25.132553 7606 tablet_bootstrap.cc:492] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: No bootstrap required, opened a new log
I20260808 04:58:25.132588 7606 ts_tablet_manager.cc:1397] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:58:25.132720 7606 raft_consensus.cc:348] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.132781 7606 raft_consensus.cc:374] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:25.132798 7606 raft_consensus.cc:729] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88c388178f74cacb3b7b5cb65039610, State: Initialized, Role: FOLLOWER
I20260808 04:58:25.132836 7606 consensus_queue.cc:260] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.132915 7606 ts_tablet_manager.cc:1428] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:25.132994 7618 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.133111 7618 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 04:58:25.134052 7052 tablet.cc:2369] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:58:25.220304 7551 tablet.cc:2369] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:25.483042 7621 raft_consensus.cc:482] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:25.483180 7621 raft_consensus.cc:504] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.483629 7621 leader_election.cc:288] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41afda761d84417683ca44c20c269049 (127.3.174.130:41423), b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841)
I20260808 04:58:25.486502 7618 raft_consensus.cc:482] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:25.486588 7618 raft_consensus.cc:504] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.486919 7618 leader_election.cc:288] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815), 41afda761d84417683ca44c20c269049 (127.3.174.130:41423)
I20260808 04:58:25.487497 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.483869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:52267 (local address 127.3.174.130:41423)
0808 04:58:25.484041 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.484043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.484053 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:25.484084 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.484085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.484096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.484159 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:25.484361 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.485130 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.486557 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.487074 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.487125 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.487136 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.487375 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.487422 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260808 04:58:25.487563 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.484035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:48787 (local address 127.3.174.131:33841)
0808 04:58:25.484091 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.484093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.484298 (+ 205us) server_negotiation.cc:407] Connection header received
0808 04:58:25.484353 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.484354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.484364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.484424 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:25.484569 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.485259 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.486635 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.487179 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.487228 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.487238 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.487464 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.487501 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:58:25.487798 7649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.483864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:33841 (local address 127.3.174.129:48787)
0808 04:58:25.484231 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.484241 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.484298 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.484427 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.484428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.484429 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:25.484500 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.484503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.485387 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.485388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.486523 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.486527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.487303 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.487304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.487380 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.487383 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.487670 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.487728 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":59,"threads_started":1}
I20260808 04:58:25.488080 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc54472868b4814a6b2d078131920f2" candidate_uuid: "17a0b77ad1554b2a84cea040d95dc482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b88c388178f74cacb3b7b5cb65039610" is_pre_election: true
I20260808 04:58:25.488166 7466 raft_consensus.cc:2454] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a0b77ad1554b2a84cea040d95dc482 in term 0.
I20260808 04:58:25.488346 6850 leader_election.cc:302] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a0b77ad1554b2a84cea040d95dc482, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:25.488498 7621 raft_consensus.cc:2781] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:25.488535 7621 raft_consensus.cc:482] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:25.488561 7621 raft_consensus.cc:3037] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:25.488783 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.483805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:41423 (local address 127.3.174.129:52267)
0808 04:58:25.483937 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.483948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.484008 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.484175 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.484176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.484179 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:25.484259 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.484261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.485249 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.485250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.486440 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.486444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.487204 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.487205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.487277 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.487282 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.488639 (+ 1357us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.488698 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260808 04:58:25.489070 7215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc54472868b4814a6b2d078131920f2" candidate_uuid: "17a0b77ad1554b2a84cea040d95dc482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afda761d84417683ca44c20c269049" is_pre_election: true
I20260808 04:58:25.489188 7215 raft_consensus.cc:2454] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a0b77ad1554b2a84cea040d95dc482 in term 0.
I20260808 04:58:25.489459 7621 raft_consensus.cc:504] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.489601 7621 leader_election.cc:288] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [CANDIDATE]: Term 1 election: Requested vote from peers 41afda761d84417683ca44c20c269049 (127.3.174.130:41423), b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841)
I20260808 04:58:25.489794 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc54472868b4814a6b2d078131920f2" candidate_uuid: "17a0b77ad1554b2a84cea040d95dc482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b88c388178f74cacb3b7b5cb65039610"
I20260808 04:58:25.489792 7215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc54472868b4814a6b2d078131920f2" candidate_uuid: "17a0b77ad1554b2a84cea040d95dc482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afda761d84417683ca44c20c269049"
I20260808 04:58:25.489863 7466 raft_consensus.cc:3037] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:25.489871 7215 raft_consensus.cc:3037] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:25.490626 7466 raft_consensus.cc:2454] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a0b77ad1554b2a84cea040d95dc482 in term 1.
I20260808 04:58:25.490738 7215 raft_consensus.cc:2454] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a0b77ad1554b2a84cea040d95dc482 in term 1.
I20260808 04:58:25.490789 6850 leader_election.cc:302] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a0b77ad1554b2a84cea040d95dc482, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:25.491118 7621 raft_consensus.cc:2781] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:25.491225 7621 raft_consensus.cc:686] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 LEADER]: Becoming Leader. State: Replica: 17a0b77ad1554b2a84cea040d95dc482, State: Running, Role: LEADER
I20260808 04:58:25.491320 7621 consensus_queue.cc:237] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.491498 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.487595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:48533 (local address 127.3.174.130:41423)
0808 04:58:25.487840 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.487841 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.488522 (+ 681us) server_negotiation.cc:407] Connection header received
0808 04:58:25.488547 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.488548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.488558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.488610 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:25.488785 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.489388 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.490609 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.491096 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.491143 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.491151 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.491405 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.491440 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260808 04:58:25.491746 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.487414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:41423 (local address 127.3.174.131:48533)
0808 04:58:25.487983 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.487994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.488052 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.488616 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.488616 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.488619 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:25.488683 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.488685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.489487 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.489488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.490496 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.490499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.491232 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.491233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.491316 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.491320 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.491636 (+ 316us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.491682 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":50,"threads_started":1}
I20260808 04:58:25.491775 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.487589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:51897 (local address 127.3.174.129:40815)
0808 04:58:25.487639 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.487640 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.487649 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:25.487686 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.487687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.487696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.487762 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:25.487865 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.488431 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.490955 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.491426 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.491491 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.491499 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.491685 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.491723 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 04:58:25.491954 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.487194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:40815 (local address 127.3.174.131:51897)
0808 04:58:25.487504 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.487514 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.487557 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.487768 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.487769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.487771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:25.487806 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.487807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.488507 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.488508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.489593 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.489596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.491508 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.491510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.491589 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.491593 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.491841 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.491890 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":57,"threads_started":1}
I20260808 04:58:25.492117 7215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc54472868b4814a6b2d078131920f2" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afda761d84417683ca44c20c269049" is_pre_election: true
I20260808 04:58:25.492159 7653 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: TxnStatusTablet state changed. Reason: New leader 17a0b77ad1554b2a84cea040d95dc482. Latest consensus state: current_term: 1 leader_uuid: "17a0b77ad1554b2a84cea040d95dc482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.492233 7653 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:58:25.492234 7215 raft_consensus.cc:2379] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b88c388178f74cacb3b7b5cb65039610 in current term 1: Already voted for candidate 17a0b77ad1554b2a84cea040d95dc482 in this term.
I20260808 04:58:25.492409 7655 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 04:58:25.492518 7655 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 04:58:25.492554 6691 catalog_manager.cc:5495] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129). New cstate: current_term: 1 leader_uuid: "17a0b77ad1554b2a84cea040d95dc482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } health_report { overall_health: UNKNOWN } } }
I20260808 04:58:25.493420 6969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc54472868b4814a6b2d078131920f2" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a0b77ad1554b2a84cea040d95dc482" is_pre_election: true
I20260808 04:58:25.493659 7350 leader_election.cc:302] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b88c388178f74cacb3b7b5cb65039610; no voters: 17a0b77ad1554b2a84cea040d95dc482, 41afda761d84417683ca44c20c269049
I20260808 04:58:25.493785 7618 raft_consensus.cc:2726] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 04:58:25.573623 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.570422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:40815 (local address 127.0.0.1:50532)
0808 04:58:25.570493 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.570507 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.570559 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.570782 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.570783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.570784 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:25.570834 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.570836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.571780 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.571781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.572452 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.572455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.572935 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.572936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.572988 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.573003 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.573414 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.573427 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.573494 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.573560 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 04:58:25.573652 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.570540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50532 (local address 127.3.174.129:40815)
0808 04:58:25.570608 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.570610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.570620 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:25.570661 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.570662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.570687 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.570782 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:25.570882 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.571636 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.572547 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.572841 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.572918 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.573103 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.573244 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.573391 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.573517 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.573543 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 04:58:25.575573 7466 raft_consensus.cc:1261] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Refusing update from remote peer 17a0b77ad1554b2a84cea040d95dc482: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:25.575865 7215 raft_consensus.cc:1261] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Refusing update from remote peer 17a0b77ad1554b2a84cea040d95dc482: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:25.575922 7653 consensus_queue.cc:1035] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:25.576186 7653 consensus_queue.cc:1035] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:25.577162 7619 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: TxnStatusTablet state changed. Reason: New leader 17a0b77ad1554b2a84cea040d95dc482. Latest consensus state: current_term: 1 leader_uuid: "17a0b77ad1554b2a84cea040d95dc482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.577265 7619 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:25.577363 7618 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: TxnStatusTablet state changed. Reason: New leader 17a0b77ad1554b2a84cea040d95dc482. Latest consensus state: current_term: 1 leader_uuid: "17a0b77ad1554b2a84cea040d95dc482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.577483 7618 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:25.578507 7653 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17a0b77ad1554b2a84cea040d95dc482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.578595 7653 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:58:25.578642 7653 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17a0b77ad1554b2a84cea040d95dc482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.578678 7653 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P 17a0b77ad1554b2a84cea040d95dc482: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:58:25.578764 7660 mvcc.cc:205] Tried to move back new op lower bound from 7316132272434864128 to 7316132272092893184. Current Snapshot: MvccSnapshot[applied={T|T < 7316132272434864128}]
I20260808 04:58:25.578859 7619 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17a0b77ad1554b2a84cea040d95dc482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.578919 7619 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P 41afda761d84417683ca44c20c269049: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:25.578964 7661 mvcc.cc:205] Tried to move back new op lower bound from 7316132272434864128 to 7316132272092893184. Current Snapshot: MvccSnapshot[applied={T|T < 7316132272434864128}]
I20260808 04:58:25.579700 7618 tablet_replica.cc:439] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17a0b77ad1554b2a84cea040d95dc482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } } }
I20260808 04:58:25.579802 7618 tablet_replica.cc:442] T bbc54472868b4814a6b2d078131920f2 P b88c388178f74cacb3b7b5cb65039610: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:25.583878 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.580994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50814 (local address 127.3.174.130:41423)
0808 04:58:25.581035 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.581037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.581117 (+ 80us) server_negotiation.cc:407] Connection header received
0808 04:58:25.581202 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.581203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.581226 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.581339 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:25.581487 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.582197 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.583045 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.583341 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.583425 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.583558 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.583674 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.583756 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.583774 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.583798 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 04:58:25.588781 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.586350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:44503 (local address 127.3.174.129:40815)
0808 04:58:25.586394 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.586396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.586450 (+ 54us) server_negotiation.cc:407] Connection header received
0808 04:58:25.586505 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.586505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.586513 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.586555 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:25.586659 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.587095 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.588139 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.588337 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.588389 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.588583 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.588607 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.588638 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.588715 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.588730 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 04:58:25.588838 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.586264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:40815 (local address 127.3.174.130:44503)
0808 04:58:25.586387 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.586398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.586459 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.586560 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.586560 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.586562 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:25.586611 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.586613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.587162 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.587163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.588057 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.588059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.588409 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.588410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.588475 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.588492 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.588656 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.588658 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.588712 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.588761 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":64,"threads_started":1}
I20260808 04:58:25.591005 6969 raft_consensus.cc:1261] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Refusing update from remote peer 41afda761d84417683ca44c20c269049: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:25.591053 7466 raft_consensus.cc:1261] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Refusing update from remote peer 41afda761d84417683ca44c20c269049: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:25.591257 7619 consensus_queue.cc:1035] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:25.591378 7678 consensus_queue.cc:1035] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:25.594054 7664 mvcc.cc:205] Tried to move back new op lower bound from 7316132272499085312 to 7316132270417567744. Current Snapshot: MvccSnapshot[applied={T|T < 7316132272499085312}]
I20260808 04:58:25.600028 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.597462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:45065 (local address 127.3.174.130:41423)
0808 04:58:25.597493 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.597495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.597700 (+ 205us) server_negotiation.cc:407] Connection header received
0808 04:58:25.597731 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.597732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.597741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.597790 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:25.597927 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.598361 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.599436 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.599619 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.599667 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.599865 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.599888 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.599917 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.599970 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.599983 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 04:58:25.600148 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.597431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:41423 (local address 127.3.174.129:45065)
0808 04:58:25.597608 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.597620 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.597692 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.597798 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.597799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.597800 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:25.597870 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.597871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.598430 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.598431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.599304 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.599307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.599714 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.599715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.599781 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.599796 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.599922 (+ 126us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.599924 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.599968 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.600009 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260808 04:58:25.609432 6691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:25.609529 6691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:25.610247 6691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353: Sending DeleteTablet for 3 replicas of tablet af89e44ee0c144b39ea8b079758946ff
I20260808 04:58:25.610599 7434 tablet_service.cc:1514] Processing DeleteTablet for tablet af89e44ee0c144b39ea8b079758946ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:25 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:41690
I20260808 04:58:25.610754 6940 tablet_service.cc:1514] Processing DeleteTablet for tablet af89e44ee0c144b39ea8b079758946ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:25 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:50524
I20260808 04:58:25.610965 7697 tablet_replica.cc:330] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: stopping tablet replica
I20260808 04:58:25.611006 6691 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 04:58:25.611097 7697 raft_consensus.cc:2229] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:25.611148 7697 raft_consensus.cc:2258] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 04:58:25.611158 6691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:25.611536 7697 ts_tablet_manager.cc:1905] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:25.611923 7186 tablet_service.cc:1514] Processing DeleteTablet for tablet af89e44ee0c144b39ea8b079758946ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:25 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:50804
I20260808 04:58:25.612910 7434 tablet_service.cc:1467] Processing CreateTablet for tablet 93a8db2205d24722afbb3a83a0688e61 (DEFAULT_TABLE table=test-table [id=7d87094f936f459cb0ff17e282955e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:25.613035 7434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93a8db2205d24722afbb3a83a0688e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:25.613683 7697 ts_tablet_manager.cc:1918] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:25.613880 7697 log.cc:1196] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/af89e44ee0c144b39ea8b079758946ff
I20260808 04:58:25.614216 7697 ts_tablet_manager.cc:1939] T af89e44ee0c144b39ea8b079758946ff P 17a0b77ad1554b2a84cea040d95dc482: Deleting consensus metadata
I20260808 04:58:25.614584 7606 tablet_bootstrap.cc:492] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: Bootstrap starting.
I20260808 04:58:25.614934 7606 tablet_bootstrap.cc:654] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:25.615545 7606 tablet_bootstrap.cc:492] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: No bootstrap required, opened a new log
I20260808 04:58:25.615592 7606 ts_tablet_manager.cc:1397] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:58:25.615648 6670 catalog_manager.cc:4841] TS 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815): tablet af89e44ee0c144b39ea8b079758946ff (table test-table [id=1e08edeb413e46f0a093db9707d23612]) successfully deleted
I20260808 04:58:25.615751 7606 raft_consensus.cc:348] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.615782 6940 tablet_service.cc:1467] Processing CreateTablet for tablet 93a8db2205d24722afbb3a83a0688e61 (DEFAULT_TABLE table=test-table [id=7d87094f936f459cb0ff17e282955e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:25.615815 7606 raft_consensus.cc:374] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:25.615834 7606 raft_consensus.cc:729] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88c388178f74cacb3b7b5cb65039610, State: Initialized, Role: FOLLOWER
I20260808 04:58:25.615871 7606 consensus_queue.cc:260] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.615960 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93a8db2205d24722afbb3a83a0688e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:25.615971 7606 ts_tablet_manager.cc:1428] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:25.616900 7609 tablet_bootstrap.cc:492] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: Bootstrap starting.
I20260808 04:58:25.617386 7609 tablet_bootstrap.cc:654] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:25.617494 7186 tablet_service.cc:1467] Processing CreateTablet for tablet 93a8db2205d24722afbb3a83a0688e61 (DEFAULT_TABLE table=test-table [id=7d87094f936f459cb0ff17e282955e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:25.617599 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93a8db2205d24722afbb3a83a0688e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:25.618002 7609 tablet_bootstrap.cc:492] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: No bootstrap required, opened a new log
I20260808 04:58:25.618047 7609 ts_tablet_manager.cc:1397] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:25.618455 7609 raft_consensus.cc:348] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.618527 7609 raft_consensus.cc:374] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:25.618546 7609 raft_consensus.cc:729] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a0b77ad1554b2a84cea040d95dc482, State: Initialized, Role: FOLLOWER
I20260808 04:58:25.618583 7609 consensus_queue.cc:260] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.618659 7609 ts_tablet_manager.cc:1428] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:25.619004 7610 tablet_bootstrap.cc:492] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: Bootstrap starting.
I20260808 04:58:25.619416 7610 tablet_bootstrap.cc:654] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:25.619812 7699 tablet_replica.cc:330] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: stopping tablet replica
I20260808 04:58:25.619922 7699 raft_consensus.cc:2229] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:25.620028 7699 raft_consensus.cc:2258] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:25.620193 7696 tablet_replica.cc:330] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: stopping tablet replica
I20260808 04:58:25.620304 7696 raft_consensus.cc:2229] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:25.620368 7696 raft_consensus.cc:2258] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:25.620548 7699 ts_tablet_manager.cc:1905] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:25.620724 7696 ts_tablet_manager.cc:1905] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:25.622309 7699 ts_tablet_manager.cc:1918] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:25.622447 7699 log.cc:1196] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/af89e44ee0c144b39ea8b079758946ff
I20260808 04:58:25.622514 7696 ts_tablet_manager.cc:1918] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:25.622576 7696 log.cc:1196] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/af89e44ee0c144b39ea8b079758946ff
I20260808 04:58:25.622823 7696 ts_tablet_manager.cc:1939] T af89e44ee0c144b39ea8b079758946ff P b88c388178f74cacb3b7b5cb65039610: Deleting consensus metadata
I20260808 04:58:25.622978 7699 ts_tablet_manager.cc:1939] T af89e44ee0c144b39ea8b079758946ff P 41afda761d84417683ca44c20c269049: Deleting consensus metadata
I20260808 04:58:25.623382 6670 catalog_manager.cc:4841] TS 41afda761d84417683ca44c20c269049 (127.3.174.130:41423): tablet af89e44ee0c144b39ea8b079758946ff (table test-table [id=1e08edeb413e46f0a093db9707d23612]) successfully deleted
I20260808 04:58:25.623524 6667 catalog_manager.cc:4841] TS b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841): tablet af89e44ee0c144b39ea8b079758946ff (table test-table [id=1e08edeb413e46f0a093db9707d23612]) successfully deleted
I20260808 04:58:25.623903 7610 tablet_bootstrap.cc:492] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: No bootstrap required, opened a new log
I20260808 04:58:25.623945 7610 ts_tablet_manager.cc:1397] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 04:58:25.624073 7610 raft_consensus.cc:348] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.624130 7610 raft_consensus.cc:374] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:25.624146 7610 raft_consensus.cc:729] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41afda761d84417683ca44c20c269049, State: Initialized, Role: FOLLOWER
I20260808 04:58:25.624186 7610 consensus_queue.cc:260] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.624270 7610 ts_tablet_manager.cc:1428] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:25.726948 7618 raft_consensus.cc:482] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:25.727052 7618 raft_consensus.cc:504] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.727209 7618 leader_election.cc:288] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815), 41afda761d84417683ca44c20c269049 (127.3.174.130:41423)
I20260808 04:58:25.727489 6969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93a8db2205d24722afbb3a83a0688e61" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a0b77ad1554b2a84cea040d95dc482" is_pre_election: true
I20260808 04:58:25.727594 6969 raft_consensus.cc:2454] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b88c388178f74cacb3b7b5cb65039610 in term 0.
I20260808 04:58:25.727572 7215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93a8db2205d24722afbb3a83a0688e61" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afda761d84417683ca44c20c269049" is_pre_election: true
I20260808 04:58:25.727679 7215 raft_consensus.cc:2454] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b88c388178f74cacb3b7b5cb65039610 in term 0.
I20260808 04:58:25.727777 7350 leader_election.cc:302] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a0b77ad1554b2a84cea040d95dc482, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:25.727883 7618 raft_consensus.cc:2781] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:25.727926 7618 raft_consensus.cc:482] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:25.727955 7618 raft_consensus.cc:3037] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:25.728956 7618 raft_consensus.cc:504] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.729065 7618 leader_election.cc:288] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 election: Requested vote from peers 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815), 41afda761d84417683ca44c20c269049 (127.3.174.130:41423)
I20260808 04:58:25.729274 6969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93a8db2205d24722afbb3a83a0688e61" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a0b77ad1554b2a84cea040d95dc482"
I20260808 04:58:25.729370 6969 raft_consensus.cc:3037] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:25.729389 7215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93a8db2205d24722afbb3a83a0688e61" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afda761d84417683ca44c20c269049"
I20260808 04:58:25.729463 7215 raft_consensus.cc:3037] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:25.730219 6969 raft_consensus.cc:2454] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b88c388178f74cacb3b7b5cb65039610 in term 1.
I20260808 04:58:25.730422 7350 leader_election.cc:302] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a0b77ad1554b2a84cea040d95dc482, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:25.730448 7215 raft_consensus.cc:2454] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b88c388178f74cacb3b7b5cb65039610 in term 1.
I20260808 04:58:25.730509 7618 raft_consensus.cc:2781] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:25.730623 7618 raft_consensus.cc:686] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 1 LEADER]: Becoming Leader. State: Replica: b88c388178f74cacb3b7b5cb65039610, State: Running, Role: LEADER
I20260808 04:58:25.730696 7618 consensus_queue.cc:237] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:25.731417 6691 catalog_manager.cc:5495] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 reported cstate change: term changed from 0 to 1, leader changed from <none> to b88c388178f74cacb3b7b5cb65039610 (127.3.174.131). New cstate: current_term: 1 leader_uuid: "b88c388178f74cacb3b7b5cb65039610" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } health_report { overall_health: HEALTHY } } }
I20260808 04:58:25.740315 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.737906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.3.174.131:33841)
0808 04:58:25.737954 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.737956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.737966 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:25.738000 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.738001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.738024 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.738104 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:25.738188 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.738916 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.739596 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.739795 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.739850 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.739995 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.740099 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.740158 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.740198 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.740221 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 04:58:25.746120 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.743282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:39611 (local address 127.3.174.129:40815)
0808 04:58:25.743331 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.743333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.743485 (+ 152us) server_negotiation.cc:407] Connection header received
0808 04:58:25.743557 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.743558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.743566 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.743607 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:25.743803 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.744230 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.745472 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.745653 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.745708 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.745927 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.745952 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.745983 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.746051 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.746068 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 04:58:25.746244 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.743179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:40815 (local address 127.3.174.131:39611)
0808 04:58:25.743426 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.743437 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.743503 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.743611 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.743612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.743614 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:25.743738 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.743740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.744296 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.744297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.745395 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.745398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.745723 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.745724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.745826 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.745845 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.746001 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.746003 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.746050 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.746094 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":136,"threads_started":1}
I20260808 04:58:25.748306 7215 raft_consensus.cc:1261] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Refusing update from remote peer b88c388178f74cacb3b7b5cb65039610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:25.748334 6969 raft_consensus.cc:1261] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Refusing update from remote peer b88c388178f74cacb3b7b5cb65039610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:25.748489 7618 consensus_queue.cc:1035] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:25.748646 7707 consensus_queue.cc:1035] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:25.756013 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.753762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:33841 (local address 127.3.174.129:52975)
0808 04:58:25.753804 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:25.753814 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:25.753854 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:25.753988 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:25.753989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:25.753991 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:25.754029 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.754031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.754778 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.754779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.755304 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:25.755306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.755618 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.755619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:25.755671 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.755695 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.755871 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.755873 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:58:25.755921 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 04:58:25.755963 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 04:58:25.756138 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:25.753856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:52975 (local address 127.3.174.131:33841)
0808 04:58:25.753896 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:25.753898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:25.753908 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:25.753927 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:25.753928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:25.753936 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:25.753980 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:25.754094 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.754709 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.755371 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:25.755564 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:25.755615 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:25.755777 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:25.755809 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:25.755848 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:25.756084 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 04:58:25.756100 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 04:58:25.756572 7664 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 04:58:27.275797 6692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:27.275898 6692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:27.276598 6692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353: Sending DeleteTablet for 3 replicas of tablet 93a8db2205d24722afbb3a83a0688e61
I20260808 04:58:27.276818 6940 tablet_service.cc:1514] Processing DeleteTablet for tablet 93a8db2205d24722afbb3a83a0688e61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:27 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:50524
I20260808 04:58:27.276849 7186 tablet_service.cc:1514] Processing DeleteTablet for tablet 93a8db2205d24722afbb3a83a0688e61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:27 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:50804
I20260808 04:58:27.276885 7434 tablet_service.cc:1514] Processing DeleteTablet for tablet 93a8db2205d24722afbb3a83a0688e61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:27 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:41690
I20260808 04:58:27.277336 7768 tablet_replica.cc:330] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: stopping tablet replica
I20260808 04:58:27.277388 7767 tablet_replica.cc:330] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: stopping tablet replica
I20260808 04:58:27.277448 7767 raft_consensus.cc:2229] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:27.277489 7767 raft_consensus.cc:2258] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:27.277580 7766 tablet_replica.cc:330] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: stopping tablet replica
I20260808 04:58:27.277658 7766 raft_consensus.cc:2229] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:27.277659 7768 raft_consensus.cc:2229] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:27.277721 7766 raft_consensus.cc:2258] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:27.277777 7768 raft_consensus.cc:2258] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:27.278017 7768 ts_tablet_manager.cc:1905] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:27.278180 7767 ts_tablet_manager.cc:1905] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:27.280488 7766 ts_tablet_manager.cc:1905] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:27.281267 7767 ts_tablet_manager.cc:1918] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:58:27.281344 7767 log.cc:1196] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/93a8db2205d24722afbb3a83a0688e61
I20260808 04:58:27.281543 7767 ts_tablet_manager.cc:1939] T 93a8db2205d24722afbb3a83a0688e61 P 41afda761d84417683ca44c20c269049: Deleting consensus metadata
I20260808 04:58:27.281876 6670 catalog_manager.cc:4841] TS 41afda761d84417683ca44c20c269049 (127.3.174.130:41423): tablet 93a8db2205d24722afbb3a83a0688e61 (table test-table [id=7d87094f936f459cb0ff17e282955e46]) successfully deleted
I20260808 04:58:27.282030 7766 ts_tablet_manager.cc:1918] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:58:27.282092 7766 log.cc:1196] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/93a8db2205d24722afbb3a83a0688e61
I20260808 04:58:27.282347 7766 ts_tablet_manager.cc:1939] T 93a8db2205d24722afbb3a83a0688e61 P 17a0b77ad1554b2a84cea040d95dc482: Deleting consensus metadata
I20260808 04:58:27.282657 6670 catalog_manager.cc:4841] TS 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815): tablet 93a8db2205d24722afbb3a83a0688e61 (table test-table [id=7d87094f936f459cb0ff17e282955e46]) successfully deleted
I20260808 04:58:27.282963 7768 ts_tablet_manager.cc:1918] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:58:27.283012 7768 log.cc:1196] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/93a8db2205d24722afbb3a83a0688e61
I20260808 04:58:27.283178 7768 ts_tablet_manager.cc:1939] T 93a8db2205d24722afbb3a83a0688e61 P b88c388178f74cacb3b7b5cb65039610: Deleting consensus metadata
I20260808 04:58:27.283560 6667 catalog_manager.cc:4841] TS b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841): tablet 93a8db2205d24722afbb3a83a0688e61 (table test-table [id=7d87094f936f459cb0ff17e282955e46]) successfully deleted
Aug 08 04:58:27 dist-test-slave-gjw7 krb5kdc[6612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165107, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 04:58:27 dist-test-slave-gjw7 krb5kdc[6612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165107, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:27.306440 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:27.296793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.3.174.190:37885)
0808 04:58:27.296966 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:27.296968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:27.296977 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:27.297000 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:27.297000 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:27.297011 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:27.297067 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:27.297675 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.298171 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.298858 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.299074 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.299122 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:27.304326 (+ 5204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:27.304327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:27.304328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:27.304334 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:27.305551 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:27.305841 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:27.305842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:27.305842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:27.305905 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:27.306052 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:27.306052 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:27.306052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:27.306148 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:27.306218 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:27.306310 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 04:58:27.306371 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260808 04:58:27.307842 6692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:27.307991 6692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:27.309265 6940 tablet_service.cc:1467] Processing CreateTablet for tablet dcf0775953f04910a818035917acfaf0 (DEFAULT_TABLE table=test-table [id=5f86dd829ea145ba89062df02937656d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:27.309305 7186 tablet_service.cc:1467] Processing CreateTablet for tablet dcf0775953f04910a818035917acfaf0 (DEFAULT_TABLE table=test-table [id=5f86dd829ea145ba89062df02937656d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:27.309336 7434 tablet_service.cc:1467] Processing CreateTablet for tablet dcf0775953f04910a818035917acfaf0 (DEFAULT_TABLE table=test-table [id=5f86dd829ea145ba89062df02937656d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:27.309435 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcf0775953f04910a818035917acfaf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:27.309437 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcf0775953f04910a818035917acfaf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:27.309464 7434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcf0775953f04910a818035917acfaf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:27.310632 7785 tablet_bootstrap.cc:492] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: Bootstrap starting.
I20260808 04:58:27.310779 7786 tablet_bootstrap.cc:492] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: Bootstrap starting.
I20260808 04:58:27.310985 7785 tablet_bootstrap.cc:654] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:27.311422 7786 tablet_bootstrap.cc:654] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:27.311619 7785 tablet_bootstrap.cc:492] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: No bootstrap required, opened a new log
I20260808 04:58:27.311697 7785 ts_tablet_manager.cc:1397] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:27.311980 7790 tablet_bootstrap.cc:492] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: Bootstrap starting.
I20260808 04:58:27.312009 7785 raft_consensus.cc:348] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.312086 7785 raft_consensus.cc:374] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:27.312143 7785 raft_consensus.cc:729] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a0b77ad1554b2a84cea040d95dc482, State: Initialized, Role: FOLLOWER
I20260808 04:58:27.312206 7785 consensus_queue.cc:260] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.312345 7785 ts_tablet_manager.cc:1428] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:27.312691 7790 tablet_bootstrap.cc:654] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:27.313277 7786 tablet_bootstrap.cc:492] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: No bootstrap required, opened a new log
I20260808 04:58:27.313354 7786 ts_tablet_manager.cc:1397] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 04:58:27.313534 7790 tablet_bootstrap.cc:492] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: No bootstrap required, opened a new log
I20260808 04:58:27.313576 7790 ts_tablet_manager.cc:1397] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:27.313580 7786 raft_consensus.cc:348] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.313657 7786 raft_consensus.cc:374] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:27.313683 7786 raft_consensus.cc:729] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88c388178f74cacb3b7b5cb65039610, State: Initialized, Role: FOLLOWER
I20260808 04:58:27.313666 7790 raft_consensus.cc:348] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.313736 7790 raft_consensus.cc:374] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:27.313760 7790 raft_consensus.cc:729] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41afda761d84417683ca44c20c269049, State: Initialized, Role: FOLLOWER
I20260808 04:58:27.313758 7786 consensus_queue.cc:260] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.313808 7790 consensus_queue.cc:260] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.313868 7786 ts_tablet_manager.cc:1428] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:27.313964 7790 ts_tablet_manager.cc:1428] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:27.371373 7653 raft_consensus.cc:482] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:27.371479 7653 raft_consensus.cc:504] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.371645 7653 leader_election.cc:288] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41afda761d84417683ca44c20c269049 (127.3.174.130:41423), b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841)
I20260808 04:58:27.371872 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcf0775953f04910a818035917acfaf0" candidate_uuid: "17a0b77ad1554b2a84cea040d95dc482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b88c388178f74cacb3b7b5cb65039610" is_pre_election: true
I20260808 04:58:27.371881 7215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcf0775953f04910a818035917acfaf0" candidate_uuid: "17a0b77ad1554b2a84cea040d95dc482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afda761d84417683ca44c20c269049" is_pre_election: true
I20260808 04:58:27.371961 7466 raft_consensus.cc:2454] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a0b77ad1554b2a84cea040d95dc482 in term 0.
I20260808 04:58:27.371978 7215 raft_consensus.cc:2454] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a0b77ad1554b2a84cea040d95dc482 in term 0.
I20260808 04:58:27.372110 6850 leader_election.cc:302] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a0b77ad1554b2a84cea040d95dc482, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:27.372251 7653 raft_consensus.cc:2781] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:27.372292 7653 raft_consensus.cc:482] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:27.372313 7653 raft_consensus.cc:3037] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:27.373085 7653 raft_consensus.cc:504] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.373219 7653 leader_election.cc:288] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [CANDIDATE]: Term 1 election: Requested vote from peers 41afda761d84417683ca44c20c269049 (127.3.174.130:41423), b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841)
I20260808 04:58:27.373399 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcf0775953f04910a818035917acfaf0" candidate_uuid: "17a0b77ad1554b2a84cea040d95dc482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b88c388178f74cacb3b7b5cb65039610"
I20260808 04:58:27.373425 7215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcf0775953f04910a818035917acfaf0" candidate_uuid: "17a0b77ad1554b2a84cea040d95dc482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afda761d84417683ca44c20c269049"
I20260808 04:58:27.373489 7215 raft_consensus.cc:3037] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:27.373489 7466 raft_consensus.cc:3037] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:27.374430 7466 raft_consensus.cc:2454] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a0b77ad1554b2a84cea040d95dc482 in term 1.
I20260808 04:58:27.374430 7215 raft_consensus.cc:2454] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a0b77ad1554b2a84cea040d95dc482 in term 1.
I20260808 04:58:27.374610 6850 leader_election.cc:302] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a0b77ad1554b2a84cea040d95dc482, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:27.374738 7653 raft_consensus.cc:2781] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:27.374785 7653 raft_consensus.cc:686] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 LEADER]: Becoming Leader. State: Replica: 17a0b77ad1554b2a84cea040d95dc482, State: Running, Role: LEADER
I20260808 04:58:27.374836 7653 consensus_queue.cc:237] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.375560 6691 catalog_manager.cc:5495] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129). New cstate: current_term: 1 leader_uuid: "17a0b77ad1554b2a84cea040d95dc482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } health_report { overall_health: UNKNOWN } } }
I20260808 04:58:27.387346 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:27.385161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.3.174.129:40815)
0808 04:58:27.385337 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:27.385339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:27.385347 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:27.385369 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:27.385370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:27.385380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:27.385434 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:27.385536 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.386080 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.386837 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.387023 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.387072 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:27.387154 (+ 82us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:27.387181 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:27.387218 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:27.387257 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 04:58:27.387273 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260808 04:58:27.391779 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:27.389767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:49281 (local address 127.3.174.129:40815)
0808 04:58:27.389803 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:27.389805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:27.389858 (+ 53us) server_negotiation.cc:407] Connection header received
0808 04:58:27.389916 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:27.389917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:27.389927 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:27.389970 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:27.390075 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.390500 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.391216 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.391384 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.391435 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:27.391541 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:27.391570 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:27.391608 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:27.391655 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 04:58:27.391700 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":19}
I20260808 04:58:27.391784 7809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:27.389625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:40815 (local address 127.3.174.129:49281)
0808 04:58:27.389805 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:27.389816 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:27.389875 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:27.389975 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:27.389976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:27.389978 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:27.390023 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:27.390025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.390562 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.390563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:27.391143 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:27.391145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.391438 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.391439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:27.391489 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:27.391492 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:27.391614 (+ 122us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:27.391616 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:58:27.391655 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 04:58:27.391711 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260808 04:58:27.393553 7215 raft_consensus.cc:1261] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Refusing update from remote peer 17a0b77ad1554b2a84cea040d95dc482: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:27.393762 7466 raft_consensus.cc:1261] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Refusing update from remote peer 17a0b77ad1554b2a84cea040d95dc482: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:27.393899 7653 consensus_queue.cc:1035] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:27.394004 7810 consensus_queue.cc:1035] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:27.404495 6692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:27.404584 6692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:27.405139 6692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353: Sending DeleteTablet for 3 replicas of tablet dcf0775953f04910a818035917acfaf0
I20260808 04:58:27.405382 6940 tablet_service.cc:1514] Processing DeleteTablet for tablet dcf0775953f04910a818035917acfaf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:27 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:50524
I20260808 04:58:27.405371 7434 tablet_service.cc:1514] Processing DeleteTablet for tablet dcf0775953f04910a818035917acfaf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:27 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:41690
I20260808 04:58:27.405465 7768 tablet_replica.cc:330] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: stopping tablet replica
I20260808 04:58:27.405464 7766 tablet_replica.cc:330] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: stopping tablet replica
I20260808 04:58:27.405514 7766 raft_consensus.cc:2229] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:27.405516 7768 raft_consensus.cc:2229] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:27.405545 7768 raft_consensus.cc:2258] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:27.405592 7766 raft_consensus.cc:2258] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:27.405767 7186 tablet_service.cc:1514] Processing DeleteTablet for tablet dcf0775953f04910a818035917acfaf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:27 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:50804
I20260808 04:58:27.405892 7767 tablet_replica.cc:330] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: stopping tablet replica
I20260808 04:58:27.405953 7767 raft_consensus.cc:2229] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:27.405995 7767 raft_consensus.cc:2258] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:27.405954 6692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:27.406082 6692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:27.406176 7766 ts_tablet_manager.cc:1905] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:27.407338 7186 tablet_service.cc:1467] Processing CreateTablet for tablet 048306f211284f2d8fc63ab354024267 (DEFAULT_TABLE table=test-table [id=a788e78cd7dd404ea307f77095fb21ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:27.407442 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 048306f211284f2d8fc63ab354024267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:27.407800 7766 ts_tablet_manager.cc:1918] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:27.407846 7766 log.cc:1196] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/dcf0775953f04910a818035917acfaf0
I20260808 04:58:27.408032 7766 ts_tablet_manager.cc:1939] T dcf0775953f04910a818035917acfaf0 P 17a0b77ad1554b2a84cea040d95dc482: Deleting consensus metadata
I20260808 04:58:27.408349 7434 tablet_service.cc:1467] Processing CreateTablet for tablet 048306f211284f2d8fc63ab354024267 (DEFAULT_TABLE table=test-table [id=a788e78cd7dd404ea307f77095fb21ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:27.408445 7434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 048306f211284f2d8fc63ab354024267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:27.409088 7790 tablet_bootstrap.cc:492] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: Bootstrap starting.
I20260808 04:58:27.409548 7790 tablet_bootstrap.cc:654] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:27.409571 6940 tablet_service.cc:1467] Processing CreateTablet for tablet 048306f211284f2d8fc63ab354024267 (DEFAULT_TABLE table=test-table [id=a788e78cd7dd404ea307f77095fb21ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:27.409664 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 048306f211284f2d8fc63ab354024267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:27.409729 7786 tablet_bootstrap.cc:492] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: Bootstrap starting.
I20260808 04:58:27.409996 7786 tablet_bootstrap.cc:654] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:27.410566 7786 tablet_bootstrap.cc:492] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: No bootstrap required, opened a new log
I20260808 04:58:27.410617 7786 ts_tablet_manager.cc:1397] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:27.410836 7786 raft_consensus.cc:348] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.410883 7767 ts_tablet_manager.cc:1905] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:27.410918 7786 raft_consensus.cc:374] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:27.410939 7786 raft_consensus.cc:729] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88c388178f74cacb3b7b5cb65039610, State: Initialized, Role: FOLLOWER
I20260808 04:58:27.410976 7786 consensus_queue.cc:260] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.411055 7786 ts_tablet_manager.cc:1428] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:27.411185 7785 tablet_bootstrap.cc:492] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482: Bootstrap starting.
I20260808 04:58:27.411227 6670 catalog_manager.cc:4841] TS 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815): tablet dcf0775953f04910a818035917acfaf0 (table test-table [id=5f86dd829ea145ba89062df02937656d]) successfully deleted
I20260808 04:58:27.411548 7785 tablet_bootstrap.cc:654] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:27.412333 7767 ts_tablet_manager.cc:1918] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:27.412384 7767 log.cc:1196] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/dcf0775953f04910a818035917acfaf0
I20260808 04:58:27.412415 7785 tablet_bootstrap.cc:492] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482: No bootstrap required, opened a new log
I20260808 04:58:27.412467 7785 ts_tablet_manager.cc:1397] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:27.412573 7767 ts_tablet_manager.cc:1939] T dcf0775953f04910a818035917acfaf0 P 41afda761d84417683ca44c20c269049: Deleting consensus metadata
I20260808 04:58:27.412561 7785 raft_consensus.cc:348] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.412628 7785 raft_consensus.cc:374] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:27.412654 7785 raft_consensus.cc:729] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a0b77ad1554b2a84cea040d95dc482, State: Initialized, Role: FOLLOWER
I20260808 04:58:27.412705 7785 consensus_queue.cc:260] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.412787 7785 ts_tablet_manager.cc:1428] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:27.412986 6670 catalog_manager.cc:4841] TS 41afda761d84417683ca44c20c269049 (127.3.174.130:41423): tablet dcf0775953f04910a818035917acfaf0 (table test-table [id=5f86dd829ea145ba89062df02937656d]) successfully deleted
I20260808 04:58:27.413429 7768 ts_tablet_manager.cc:1905] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:27.414876 7768 ts_tablet_manager.cc:1918] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:27.414933 7768 log.cc:1196] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/dcf0775953f04910a818035917acfaf0
I20260808 04:58:27.415119 7768 ts_tablet_manager.cc:1939] T dcf0775953f04910a818035917acfaf0 P b88c388178f74cacb3b7b5cb65039610: Deleting consensus metadata
I20260808 04:58:27.415369 7790 tablet_bootstrap.cc:492] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: No bootstrap required, opened a new log
I20260808 04:58:27.415416 7790 ts_tablet_manager.cc:1397] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260808 04:58:27.415459 6667 catalog_manager.cc:4841] TS b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841): tablet dcf0775953f04910a818035917acfaf0 (table test-table [id=5f86dd829ea145ba89062df02937656d]) successfully deleted
I20260808 04:58:27.415512 7790 raft_consensus.cc:348] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.415570 7790 raft_consensus.cc:374] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:27.415586 7790 raft_consensus.cc:729] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41afda761d84417683ca44c20c269049, State: Initialized, Role: FOLLOWER
I20260808 04:58:27.415630 7790 consensus_queue.cc:260] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.415722 7790 ts_tablet_manager.cc:1428] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:27.563824 7751 raft_consensus.cc:482] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:27.563934 7751 raft_consensus.cc:504] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.564107 7751 leader_election.cc:288] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815), 41afda761d84417683ca44c20c269049 (127.3.174.130:41423)
I20260808 04:58:27.564375 6969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "048306f211284f2d8fc63ab354024267" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a0b77ad1554b2a84cea040d95dc482" is_pre_election: true
I20260808 04:58:27.564378 7215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "048306f211284f2d8fc63ab354024267" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afda761d84417683ca44c20c269049" is_pre_election: true
I20260808 04:58:27.564473 6969 raft_consensus.cc:2454] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b88c388178f74cacb3b7b5cb65039610 in term 0.
I20260808 04:58:27.564489 7215 raft_consensus.cc:2454] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b88c388178f74cacb3b7b5cb65039610 in term 0.
I20260808 04:58:27.564647 7350 leader_election.cc:302] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41afda761d84417683ca44c20c269049, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:27.564757 7751 raft_consensus.cc:2781] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:27.564824 7751 raft_consensus.cc:482] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:27.564843 7751 raft_consensus.cc:3037] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:27.565604 7751 raft_consensus.cc:504] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.565716 7751 leader_election.cc:288] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 election: Requested vote from peers 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815), 41afda761d84417683ca44c20c269049 (127.3.174.130:41423)
I20260808 04:58:27.565898 6969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "048306f211284f2d8fc63ab354024267" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a0b77ad1554b2a84cea040d95dc482"
I20260808 04:58:27.565927 7215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "048306f211284f2d8fc63ab354024267" candidate_uuid: "b88c388178f74cacb3b7b5cb65039610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afda761d84417683ca44c20c269049"
I20260808 04:58:27.565973 6969 raft_consensus.cc:3037] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:27.565984 7215 raft_consensus.cc:3037] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:27.566574 7215 raft_consensus.cc:2454] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b88c388178f74cacb3b7b5cb65039610 in term 1.
I20260808 04:58:27.566741 7350 leader_election.cc:302] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41afda761d84417683ca44c20c269049, b88c388178f74cacb3b7b5cb65039610; no voters:
I20260808 04:58:27.566766 6969 raft_consensus.cc:2454] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b88c388178f74cacb3b7b5cb65039610 in term 1.
I20260808 04:58:27.566838 7751 raft_consensus.cc:2781] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:27.566895 7751 raft_consensus.cc:686] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 1 LEADER]: Becoming Leader. State: Replica: b88c388178f74cacb3b7b5cb65039610, State: Running, Role: LEADER
I20260808 04:58:27.566939 7751 consensus_queue.cc:237] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } }
I20260808 04:58:27.567513 6691 catalog_manager.cc:5495] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 reported cstate change: term changed from 0 to 1, leader changed from <none> to b88c388178f74cacb3b7b5cb65039610 (127.3.174.131). New cstate: current_term: 1 leader_uuid: "b88c388178f74cacb3b7b5cb65039610" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b88c388178f74cacb3b7b5cb65039610" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33841 } health_report { overall_health: HEALTHY } } }
I20260808 04:58:27.593281 7826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:27.590932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41712 (local address 127.3.174.131:33841)
0808 04:58:27.591292 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:27.591294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:27.591301 (+ 7us) server_negotiation.cc:407] Connection header received
0808 04:58:27.591320 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:27.591320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:27.591330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:27.591385 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:27.591540 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.592005 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.592720 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:27.592896 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:27.592937 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:27.593067 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:27.593092 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:27.593123 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:27.593152 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 04:58:27.593168 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":265,"threads_started":1}
I20260808 04:58:27.595314 6969 raft_consensus.cc:1261] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482 [term 1 FOLLOWER]: Refusing update from remote peer b88c388178f74cacb3b7b5cb65039610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:27.595327 7215 raft_consensus.cc:1261] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Refusing update from remote peer b88c388178f74cacb3b7b5cb65039610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:27.595510 7759 consensus_queue.cc:1035] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a0b77ad1554b2a84cea040d95dc482" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 40815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:27.595615 7751 consensus_queue.cc:1035] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41afda761d84417683ca44c20c269049" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 41423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 04:58:27.599836 7732 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 04:58:29.116917 6688 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:29.117081 6688 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:29.120566 6688 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd7d192ecb8344838a0ab3a9e6575353: Sending DeleteTablet for 3 replicas of tablet 048306f211284f2d8fc63ab354024267
I20260808 04:58:29.120831 6939 tablet_service.cc:1514] Processing DeleteTablet for tablet 048306f211284f2d8fc63ab354024267 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:29 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:50524
I20260808 04:58:29.120884 7186 tablet_service.cc:1514] Processing DeleteTablet for tablet 048306f211284f2d8fc63ab354024267 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:29 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:50804
I20260808 04:58:29.120952 7434 tablet_service.cc:1514] Processing DeleteTablet for tablet 048306f211284f2d8fc63ab354024267 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:29 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:41690
I20260808 04:58:29.121061 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 6806
I20260808 04:58:29.121099 7879 tablet_replica.cc:330] T 048306f211284f2d8fc63ab354024267 P 17a0b77ad1554b2a84cea040d95dc482: stopping tablet replica
I20260808 04:58:29.125367 7880 tablet_replica.cc:330] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: stopping tablet replica
I20260808 04:58:29.125437 7880 raft_consensus.cc:2229] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:29.125500 7880 raft_consensus.cc:2258] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:29.126784 7880 ts_tablet_manager.cc:1905] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:29.128980 7878 tablet_replica.cc:330] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: stopping tablet replica
I20260808 04:58:29.129048 7878 raft_consensus.cc:2229] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:29.129091 7878 raft_consensus.cc:2258] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:29.131186 7878 ts_tablet_manager.cc:1905] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:29.132467 7878 ts_tablet_manager.cc:1918] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:58:29.132522 7878 log.cc:1196] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/048306f211284f2d8fc63ab354024267
I20260808 04:58:29.132756 7878 ts_tablet_manager.cc:1939] T 048306f211284f2d8fc63ab354024267 P 41afda761d84417683ca44c20c269049: Deleting consensus metadata
I20260808 04:58:29.133095 6670 catalog_manager.cc:4841] TS 41afda761d84417683ca44c20c269049 (127.3.174.130:41423): tablet 048306f211284f2d8fc63ab354024267 (table test-table [id=a788e78cd7dd404ea307f77095fb21ab]) successfully deleted
W20260808 04:58:29.137712 6670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:40815 (error 108)
I20260808 04:58:29.137965 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 7061
I20260808 04:58:29.140489 7884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:29.138088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:40815 (local address 127.0.0.1:50548)
0808 04:58:29.140394 (+ 2306us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:29.140451 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:40815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2274,"thread_start_us":60,"threads_started":1}
W20260808 04:58:29.140602 6670 catalog_manager.cc:4584] TS 17a0b77ad1554b2a84cea040d95dc482 (127.3.174.129:40815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 048306f211284f2d8fc63ab354024267: Network error: Client connection negotiation failed: client connection to 127.3.174.129:40815: connect: Connection refused (error 111)
I20260808 04:58:29.138023 7880 ts_tablet_manager.cc:1918] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 04:58:29.142148 7880 log.cc:1196] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/048306f211284f2d8fc63ab354024267
I20260808 04:58:29.142354 7880 ts_tablet_manager.cc:1939] T 048306f211284f2d8fc63ab354024267 P b88c388178f74cacb3b7b5cb65039610: Deleting consensus metadata
I20260808 04:58:29.142679 6667 catalog_manager.cc:4841] TS b88c388178f74cacb3b7b5cb65039610 (127.3.174.131:33841): tablet 048306f211284f2d8fc63ab354024267 (table test-table [id=a788e78cd7dd404ea307f77095fb21ab]) successfully deleted
I20260808 04:58:29.155925 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 7318
I20260808 04:58:29.168633 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 6640
2026-08-08T04:58:29Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6144 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:29 dist-test-slave-gjw7 krb5kdc[7890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:29 dist-test-slave-gjw7 krb5kdc[7890](info): set up 2 sockets
Aug 08 04:58:29 dist-test-slave-gjw7 krb5kdc[7890](info): commencing operation
krb5kdc: starting...
W20260808 04:58:30.361150 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:30 dist-test-slave-gjw7 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:30.480086 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46343
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:46343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:30.562152 7919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:30.562350 7919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:30.562378 7919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:30.563494 7919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:30.563535 7919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:30.563555 7919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:30.563570 7919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:30.563584 7919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:30.565037 7919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:46343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:30.565271 7919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:30.565486 7919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:58:30.570420 7919 server_base.cc:1034] running on GCE node
W20260808 04:58:30.570399 7929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:30.570398 7926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:30.570392 7927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:30.571457 7919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:30.571738 7919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:30.572881 7919 hybrid_clock.cc:648] HybridClock initialized: now 1786165110572868 us; error 28 us; skew 500 ppm
Aug 08 04:58:30 dist-test-slave-gjw7 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165110, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:30.575381 7919 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:30.575764 7919 webserver.cc:460] Webserver started at http://127.3.174.190:38647/ using document root <none> and password file <none>
I20260808 04:58:30.575951 7919 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:30.575992 7919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:30.576077 7919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:30.577046 7919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "5f98455d860e4ac8bf881b61d76a15a2"
format_stamp: "Formatted at 2026-08-08 04:58:30 on dist-test-slave-gjw7"
server_key: "68b68ea20c3d6547fd0899ac21e22b78"
server_key_iv: "c36ac983ab19842e126aa7e4e7d7f185"
server_key_version: "encryptionkey@0"
I20260808 04:58:30.577378 7919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "5f98455d860e4ac8bf881b61d76a15a2"
format_stamp: "Formatted at 2026-08-08 04:58:30 on dist-test-slave-gjw7"
server_key: "68b68ea20c3d6547fd0899ac21e22b78"
server_key_iv: "c36ac983ab19842e126aa7e4e7d7f185"
server_key_version: "encryptionkey@0"
I20260808 04:58:30.578627 7919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:30.579286 7944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:30.579445 7919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:30.579528 7919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "5f98455d860e4ac8bf881b61d76a15a2"
format_stamp: "Formatted at 2026-08-08 04:58:30 on dist-test-slave-gjw7"
server_key: "68b68ea20c3d6547fd0899ac21e22b78"
server_key_iv: "c36ac983ab19842e126aa7e4e7d7f185"
server_key_version: "encryptionkey@0"
I20260808 04:58:30.579582 7919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:30.589246 7919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:30.590903 7919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:30.591004 7919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:30.595394 7919 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:46343
I20260808 04:58:30.595430 8037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:46343 every 8 connection(s)
I20260808 04:58:30.595799 7919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:30.596663 8039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:30.598977 8039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2: Bootstrap starting.
I20260808 04:58:30.599521 8039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:30.599779 8039 log.cc:824] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:30.600407 8039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2: No bootstrap required, opened a new log
I20260808 04:58:30.602031 8039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f98455d860e4ac8bf881b61d76a15a2" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46343 } }
I20260808 04:58:30.602165 8039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:30.602197 8039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f98455d860e4ac8bf881b61d76a15a2, State: Initialized, Role: FOLLOWER
I20260808 04:58:30.602327 8039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f98455d860e4ac8bf881b61d76a15a2" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46343 } }
I20260808 04:58:30.602408 8039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:30.602442 8039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:30.602489 8039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:30.602974 8039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f98455d860e4ac8bf881b61d76a15a2" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46343 } }
I20260808 04:58:30.603073 8039 leader_election.cc:302] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f98455d860e4ac8bf881b61d76a15a2; no voters:
I20260808 04:58:30.603238 8039 leader_election.cc:288] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:30.603286 8042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:30.603482 8039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:30.603597 8042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [term 1 LEADER]: Becoming Leader. State: Replica: 5f98455d860e4ac8bf881b61d76a15a2, State: Running, Role: LEADER
I20260808 04:58:30.603693 8042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f98455d860e4ac8bf881b61d76a15a2" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46343 } }
I20260808 04:58:30.604055 8043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f98455d860e4ac8bf881b61d76a15a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f98455d860e4ac8bf881b61d76a15a2" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46343 } } }
I20260808 04:58:30.604149 8043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:30.604315 8045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f98455d860e4ac8bf881b61d76a15a2. Latest consensus state: current_term: 1 leader_uuid: "5f98455d860e4ac8bf881b61d76a15a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f98455d860e4ac8bf881b61d76a15a2" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46343 } } }
I20260808 04:58:30.604373 8045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:30.604368 8054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:30.604871 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 7919
I20260808 04:58:30.604969 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:30.605118 3770 external_mini_cluster.cc:1428] Setting key 429ca48826174f6dd722b3860bc80152
I20260808 04:58:30.605136 8054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:30.606652 8054 catalog_manager.cc:1347] Generated new cluster ID: 05740eb442c24fbcb29d083baa33c230
I20260808 04:58:30.606704 8054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 04:58:30 dist-test-slave-gjw7 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:30.620815 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:30.605690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.3.174.190:46343)
0808 04:58:30.609417 (+ 3727us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:30.609427 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:30.609462 (+ 35us) server_negotiation.cc:407] Connection header received
0808 04:58:30.609585 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:30.609593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:30.609650 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:30.609792 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:30.610523 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.612880 (+ 2357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.613925 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.614562 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.614673 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:30.617408 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:30.617413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:30.617413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:30.617425 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:30.619225 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:30.619603 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:30.619605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:30.619606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:30.619721 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:30.619899 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:30.619900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:30.619900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:30.620194 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:30.620330 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:30.620444 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 04:58:30.620571 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3670,"thread_start_us":66,"threads_started":1}
I20260808 04:58:30.635975 8054 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:30.636677 8054 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:30.643599 8054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f98455d860e4ac8bf881b61d76a15a2: Generated new TSK 0
I20260808 04:58:30.643821 8054 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:30.674167 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46343
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:30.756328 8083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:30.756520 8083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:30.756542 8083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:30.757704 8083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:30.757743 8083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:30.757768 8083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:30.759210 8083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:30.759426 8083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:30.759632 8083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:30.764413 8096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:30.764424 8092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:30.764488 8083 server_base.cc:1034] running on GCE node
W20260808 04:58:30.764451 8090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:30.764854 8083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:30.765069 8083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:30.766237 8083 hybrid_clock.cc:648] HybridClock initialized: now 1786165110766221 us; error 29 us; skew 500 ppm
Aug 08 04:58:30 dist-test-slave-gjw7 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165110, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:30.768821 8083 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:30.769240 8083 webserver.cc:460] Webserver started at http://127.3.174.129:40967/ using document root <none> and password file <none>
I20260808 04:58:30.769455 8083 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:30.769505 8083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:30.769606 8083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:30.770589 8083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "55db5c09bf2047208f020890c85be789"
format_stamp: "Formatted at 2026-08-08 04:58:30 on dist-test-slave-gjw7"
server_key: "8ff1c11745f19ff3785246d5547262ba"
server_key_iv: "bc0ff3c012237b47709f6dacd6211f20"
server_key_version: "encryptionkey@0"
I20260808 04:58:30.770910 8083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "55db5c09bf2047208f020890c85be789"
format_stamp: "Formatted at 2026-08-08 04:58:30 on dist-test-slave-gjw7"
server_key: "8ff1c11745f19ff3785246d5547262ba"
server_key_iv: "bc0ff3c012237b47709f6dacd6211f20"
server_key_version: "encryptionkey@0"
I20260808 04:58:30.772253 8083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:30.772922 8108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:30.773068 8083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:30.773146 8083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "55db5c09bf2047208f020890c85be789"
format_stamp: "Formatted at 2026-08-08 04:58:30 on dist-test-slave-gjw7"
server_key: "8ff1c11745f19ff3785246d5547262ba"
server_key_iv: "bc0ff3c012237b47709f6dacd6211f20"
server_key_version: "encryptionkey@0"
I20260808 04:58:30.773216 8083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:30.780408 8083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:30.782460 8083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:30.782577 8083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:30.782771 8083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:30.783118 8083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:30.783161 8083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:30.783181 8083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:30.783192 8083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:30.789016 8083 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:42095
I20260808 04:58:30.789088 8306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:42095 every 8 connection(s)
I20260808 04:58:30.789422 8083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 04:58:30 dist-test-slave-gjw7 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165110, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:30.798123 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:30.790168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:55915 (local address 127.3.174.190:46343)
0808 04:58:30.790225 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:30.790227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:30.790476 (+ 249us) server_negotiation.cc:407] Connection header received
0808 04:58:30.790987 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:30.790989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:30.791006 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:30.791055 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:30.792032 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.792540 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.793528 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.793750 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.793805 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:30.795817 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:30.795818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:30.795819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:30.795826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:30.796993 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:30.797396 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:30.797396 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:30.797397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:30.797447 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:30.797632 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:30.797632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:30.797632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:30.797728 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:30.797773 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:30.797987 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 04:58:30.798056 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 04:58:30.798323 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:30.790149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46343 (local address 127.3.174.129:55915)
0808 04:58:30.790353 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:30.790396 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:30.790875 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:30.791151 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:30.791155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:30.791737 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:30.791934 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:30.791939 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.792625 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.792626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:30.793450 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:30.793453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.793818 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.793819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:30.793943 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:30.794363 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:30.794372 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:30.795724 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:30.797070 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:30.797077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:30.797079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:30.797333 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:30.797496 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:30.797496 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:30.797496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:30.797590 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:30.797770 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:30.797773 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:30.797891 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 04:58:30.797968 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 04:58:30.798084 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260808 04:58:30.798808 8307 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46343
I20260808 04:58:30.798928 8307 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:30.799058 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 8083
I20260808 04:58:30.799115 8307 heartbeater.cc:511] Master 127.3.174.190:46343 requested a full tablet report, sending...
I20260808 04:58:30.799134 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:30.799356 3770 external_mini_cluster.cc:1428] Setting key a5dbeb3d6fdbb5d952786cff7e584890
I20260808 04:58:30.799712 7976 ts_manager.cc:194] Registered new tserver with Master: 55db5c09bf2047208f020890c85be789 (127.3.174.129:42095)
I20260808 04:58:30.800406 7976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:55915
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:30.824597 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46343
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:30.908282 8319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:30.908483 8319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:30.908504 8319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:30.909687 8319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:30.909729 8319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:30.909754 8319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:30.911136 8319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:30.911321 8319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:30.911506 8319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:30.916225 8330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:30.916275 8319 server_base.cc:1034] running on GCE node
W20260808 04:58:30.916198 8327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:30.916189 8328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:30.916692 8319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:30.916919 8319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:30.918056 8319 hybrid_clock.cc:648] HybridClock initialized: now 1786165110918035 us; error 40 us; skew 500 ppm
Aug 08 04:58:30 dist-test-slave-gjw7 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165110, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:30.920467 8319 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:30.920921 8319 webserver.cc:460] Webserver started at http://127.3.174.130:34221/ using document root <none> and password file <none>
I20260808 04:58:30.921124 8319 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:30.921180 8319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:30.921339 8319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:30.922415 8319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "8d549d37506a4215998031024025eb34"
format_stamp: "Formatted at 2026-08-08 04:58:30 on dist-test-slave-gjw7"
server_key: "8a37c7544651e49663ee902e0717643a"
server_key_iv: "e1f6d426b848f81c0e63c55bee24a016"
server_key_version: "encryptionkey@0"
I20260808 04:58:30.922816 8319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "8d549d37506a4215998031024025eb34"
format_stamp: "Formatted at 2026-08-08 04:58:30 on dist-test-slave-gjw7"
server_key: "8a37c7544651e49663ee902e0717643a"
server_key_iv: "e1f6d426b848f81c0e63c55bee24a016"
server_key_version: "encryptionkey@0"
I20260808 04:58:30.924156 8319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:30.924866 8338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:30.925046 8319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:30.925138 8319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "8d549d37506a4215998031024025eb34"
format_stamp: "Formatted at 2026-08-08 04:58:30 on dist-test-slave-gjw7"
server_key: "8a37c7544651e49663ee902e0717643a"
server_key_iv: "e1f6d426b848f81c0e63c55bee24a016"
server_key_version: "encryptionkey@0"
I20260808 04:58:30.925205 8319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:30.944190 8319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:30.945998 8319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:30.946147 8319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:30.946370 8319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:30.946693 8319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:30.946732 8319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:30.946766 8319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:30.946792 8319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:30.953192 8319 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:44393
I20260808 04:58:30.953249 8537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:44393 every 8 connection(s)
I20260808 04:58:30.953604 8319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 04:58:30 dist-test-slave-gjw7 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165110, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:30.959697 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 8319
I20260808 04:58:30.959776 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:30.959910 3770 external_mini_cluster.cc:1428] Setting key a01ded7e6c7bcebc49c4ba042d3d4e10
I20260808 04:58:30.963152 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:30.954308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:51799 (local address 127.3.174.190:46343)
0808 04:58:30.954374 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:30.954377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:30.954574 (+ 197us) server_negotiation.cc:407] Connection header received
0808 04:58:30.955095 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:30.955096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:30.955112 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:30.955168 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:30.956164 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.956680 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.957711 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.957909 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.957969 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:30.960427 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:30.960428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:30.960428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:30.960435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:30.961829 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:30.962294 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:30.962295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:30.962296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:30.962363 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:30.962581 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:30.962581 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:30.962581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:30.962710 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:30.962772 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:30.962980 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 04:58:30.963064 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:58:30.963217 8540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:30.954279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46343 (local address 127.3.174.130:51799)
0808 04:58:30.954454 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:30.954492 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:30.954990 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:30.955262 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:30.955267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:30.955848 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:30.956061 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:30.956067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.956775 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.956776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:30.957626 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:30.957630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:30.957994 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:30.957995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:30.958113 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:30.958538 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:30.958547 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:30.960349 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:30.961902 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:30.961909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:30.961912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:30.962202 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:30.962433 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:30.962434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:30.962434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:30.962529 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:30.962764 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:30.962767 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:30.962869 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 04:58:30.962944 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 04:58:30.963056 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260808 04:58:30.963706 8538 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46343
I20260808 04:58:30.963804 8538 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:30.963980 8538 heartbeater.cc:511] Master 127.3.174.190:46343 requested a full tablet report, sending...
I20260808 04:58:30.964337 7976 ts_manager.cc:194] Registered new tserver with Master: 8d549d37506a4215998031024025eb34 (127.3.174.130:44393)
I20260808 04:58:30.964691 7976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:51799
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:30.983520 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46343
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:31.064011 8545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:31.064205 8545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:31.064226 8545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:31.065425 8545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:31.065464 8545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:31.065490 8545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:31.067232 8545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:31.067468 8545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:31.067706 8545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:31.072562 8561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:31.072551 8557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:31.072538 8558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:31.072600 8545 server_base.cc:1034] running on GCE node
I20260808 04:58:31.072893 8545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:31.073099 8545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:31.074203 8545 hybrid_clock.cc:648] HybridClock initialized: now 1786165111074192 us; error 27 us; skew 500 ppm
Aug 08 04:58:31 dist-test-slave-gjw7 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165111, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:31.076719 8545 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:31.077140 8545 webserver.cc:460] Webserver started at http://127.3.174.131:46221/ using document root <none> and password file <none>
I20260808 04:58:31.077378 8545 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:31.077433 8545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:31.077567 8545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:31.078552 8545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "b308c6c60caf4c0cba2f06743ce36981"
format_stamp: "Formatted at 2026-08-08 04:58:31 on dist-test-slave-gjw7"
server_key: "f15f7642ee94ebf4fe69632eaee3a029"
server_key_iv: "e1f9a24754e764eda1d0582e96170567"
server_key_version: "encryptionkey@0"
I20260808 04:58:31.078843 8545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "b308c6c60caf4c0cba2f06743ce36981"
format_stamp: "Formatted at 2026-08-08 04:58:31 on dist-test-slave-gjw7"
server_key: "f15f7642ee94ebf4fe69632eaee3a029"
server_key_iv: "e1f9a24754e764eda1d0582e96170567"
server_key_version: "encryptionkey@0"
I20260808 04:58:31.079993 8545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:31.080603 8575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:31.080730 8545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:31.080811 8545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "b308c6c60caf4c0cba2f06743ce36981"
format_stamp: "Formatted at 2026-08-08 04:58:31 on dist-test-slave-gjw7"
server_key: "f15f7642ee94ebf4fe69632eaee3a029"
server_key_iv: "e1f9a24754e764eda1d0582e96170567"
server_key_version: "encryptionkey@0"
I20260808 04:58:31.080875 8545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:31.091177 8545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:31.093148 8545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:31.093269 8545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:31.093513 8545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:31.093806 8545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:31.093840 8545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:31.093868 8545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:31.093897 8545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:31.099520 8545 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:44511
I20260808 04:58:31.099562 8770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:44511 every 8 connection(s)
I20260808 04:58:31.099849 8545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 04:58:31 dist-test-slave-gjw7 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165111, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:31.107966 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 8545
I20260808 04:58:31.108072 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:31.108184 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:31.100588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:41671 (local address 127.3.174.190:46343)
0808 04:58:31.100643 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:31.100645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:31.100792 (+ 147us) server_negotiation.cc:407] Connection header received
0808 04:58:31.101269 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:31.101270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:31.101282 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:31.101369 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:31.102228 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:31.102712 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:31.103757 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:31.103938 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:31.103982 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:31.105968 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:31.105969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:31.105969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:31.105975 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:31.107068 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:31.107413 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:31.107414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:31.107414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:31.107460 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:31.107631 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:31.107632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:31.107632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:31.107721 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:31.107774 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:31.108005 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 04:58:31.108099 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 04:58:31.108285 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:31.100519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46343 (local address 127.3.174.131:41671)
0808 04:58:31.100678 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:31.100717 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:31.101164 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:31.101447 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:31.101451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:31.101974 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:31.102156 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:31.102161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:31.102786 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:31.102787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:31.103674 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:31.103677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:31.103990 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:31.103991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:31.104086 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:31.104486 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:31.104495 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:31.105889 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:31.107133 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:31.107139 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:31.107141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:31.107340 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:31.107503 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:31.107503 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:31.107504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:31.107592 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:31.107771 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:31.107774 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:31.107892 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 04:58:31.107967 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 04:58:31.108100 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260808 04:58:31.108314 3770 external_mini_cluster.cc:1428] Setting key db755c68c4bec1ded443490484c98a03
I20260808 04:58:31.108770 8773 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46343
I20260808 04:58:31.108867 8773 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:31.109028 8773 heartbeater.cc:511] Master 127.3.174.190:46343 requested a full tablet report, sending...
I20260808 04:58:31.109449 7975 ts_manager.cc:194] Registered new tserver with Master: b308c6c60caf4c0cba2f06743ce36981 (127.3.174.131:44511)
I20260808 04:58:31.109803 7975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:41671
I20260808 04:58:31.119778 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 04:58:31.131552 8073 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:31.130768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.3.174.190:46343)
0808 04:58:31.130819 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:31.130821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:31.130914 (+ 93us) server_negotiation.cc:407] Connection header received
0808 04:58:31.130962 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:31.130963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:31.130971 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:31.131019 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:31.131487 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59268: BlockingRecv error: recv got EOF from 127.0.0.1:59268 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 04:58:31.132481 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 8083
I20260808 04:58:31.141757 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 8319
I20260808 04:58:31.152434 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 8545
I20260808 04:58:31.160310 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 7919
2026-08-08T04:58:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1981 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:31 dist-test-slave-gjw7 krb5kdc[8797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:31 dist-test-slave-gjw7 krb5kdc[8797](info): set up 2 sockets
Aug 08 04:58:31 dist-test-slave-gjw7 krb5kdc[8797](info): commencing operation
krb5kdc: starting...
W20260808 04:58:32.324309 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:32 dist-test-slave-gjw7 krb5kdc[8797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:32.443796 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33143
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:33143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:32.526067 8819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:32.526285 8819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:32.526315 8819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:32.527493 8819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:32.527534 8819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:32.527551 8819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:32.527566 8819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:32.527580 8819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:32.529070 8819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:33143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:32.529335 8819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:32.529549 8819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:32.534096 8827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:32.534086 8825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:32.534132 8832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:32.534479 8819 server_base.cc:1034] running on GCE node
I20260808 04:58:32.534685 8819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:32.534916 8819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:32.536048 8819 hybrid_clock.cc:648] HybridClock initialized: now 1786165112536035 us; error 28 us; skew 500 ppm
Aug 08 04:58:32 dist-test-slave-gjw7 krb5kdc[8797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165112, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:32.541499 8819 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:32.541894 8819 webserver.cc:460] Webserver started at http://127.3.174.190:43895/ using document root <none> and password file <none>
I20260808 04:58:32.542092 8819 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:32.542132 8819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:32.542228 8819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:32.543288 8819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "a4bc4389fd084bdca4081627da918c11"
format_stamp: "Formatted at 2026-08-08 04:58:32 on dist-test-slave-gjw7"
server_key: "f5bd3ec39c2f85d66123a546ec8e2479"
server_key_iv: "8c680c0b864f60a71ea8c694ab8c3c7c"
server_key_version: "encryptionkey@0"
I20260808 04:58:32.543576 8819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "a4bc4389fd084bdca4081627da918c11"
format_stamp: "Formatted at 2026-08-08 04:58:32 on dist-test-slave-gjw7"
server_key: "f5bd3ec39c2f85d66123a546ec8e2479"
server_key_iv: "8c680c0b864f60a71ea8c694ab8c3c7c"
server_key_version: "encryptionkey@0"
I20260808 04:58:32.544760 8819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:32.545452 8843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:32.545598 8819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:32.545684 8819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "a4bc4389fd084bdca4081627da918c11"
format_stamp: "Formatted at 2026-08-08 04:58:32 on dist-test-slave-gjw7"
server_key: "f5bd3ec39c2f85d66123a546ec8e2479"
server_key_iv: "8c680c0b864f60a71ea8c694ab8c3c7c"
server_key_version: "encryptionkey@0"
I20260808 04:58:32.545758 8819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:32.553740 8819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:32.555361 8819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:32.555464 8819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:32.559180 8819 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:33143
I20260808 04:58:32.559227 8930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:33143 every 8 connection(s)
I20260808 04:58:32.559525 8819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:32.560354 8931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:32.562685 8931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11: Bootstrap starting.
I20260808 04:58:32.563246 8931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:32.563519 8931 log.cc:824] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:32.564153 8931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11: No bootstrap required, opened a new log
I20260808 04:58:32.565753 8931 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bc4389fd084bdca4081627da918c11" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33143 } }
I20260808 04:58:32.565891 8931 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:32.565923 8931 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4bc4389fd084bdca4081627da918c11, State: Initialized, Role: FOLLOWER
I20260808 04:58:32.566062 8931 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bc4389fd084bdca4081627da918c11" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33143 } }
I20260808 04:58:32.566126 8931 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:32.566156 8931 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:32.566205 8931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:32.566977 8931 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bc4389fd084bdca4081627da918c11" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33143 } }
I20260808 04:58:32.567077 8931 leader_election.cc:302] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4bc4389fd084bdca4081627da918c11; no voters:
I20260808 04:58:32.567246 8931 leader_election.cc:288] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:32.567317 8934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:32.567468 8934 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [term 1 LEADER]: Becoming Leader. State: Replica: a4bc4389fd084bdca4081627da918c11, State: Running, Role: LEADER
I20260808 04:58:32.567502 8931 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:32.567596 8934 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bc4389fd084bdca4081627da918c11" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33143 } }
I20260808 04:58:32.567955 8935 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4bc4389fd084bdca4081627da918c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bc4389fd084bdca4081627da918c11" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33143 } } }
I20260808 04:58:32.567956 8938 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4bc4389fd084bdca4081627da918c11. Latest consensus state: current_term: 1 leader_uuid: "a4bc4389fd084bdca4081627da918c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4bc4389fd084bdca4081627da918c11" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33143 } } }
I20260808 04:58:32.568090 8935 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:32.568125 8938 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:32.568578 8954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:32.568702 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 8819
I20260808 04:58:32.568792 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:32.568977 3770 external_mini_cluster.cc:1428] Setting key df9714e9b605affc4b098f6cc6a40e53
I20260808 04:58:32.569047 8954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:32.570693 8954 catalog_manager.cc:1347] Generated new cluster ID: 844dfaa0863f45da9953a867b043b5e1
I20260808 04:58:32.570742 8954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 04:58:32 dist-test-slave-gjw7 krb5kdc[8797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:32.580436 8954 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:32.581095 8954 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:32.583664 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:32.569448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.3.174.190:33143)
0808 04:58:32.573098 (+ 3650us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:32.573108 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:32.573418 (+ 310us) server_negotiation.cc:407] Connection header received
0808 04:58:32.573566 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:32.573574 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:32.573629 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:32.573765 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:32.574529 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:32.575949 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:32.577036 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:32.577660 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:32.577761 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:32.580598 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:32.580603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:32.580603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:32.580614 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:32.582171 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:32.582492 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:32.582495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:32.582496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:32.582586 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:32.582785 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:32.582786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:32.582786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:32.583000 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:32.583126 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:32.583246 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 04:58:32.583492 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3583,"thread_start_us":83,"threads_started":1}
I20260808 04:58:32.584476 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:58:32.588579 8954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4bc4389fd084bdca4081627da918c11: Generated new TSK 0
I20260808 04:58:32.588743 8954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:58:32.601099 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:32.597513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38092 (local address 127.3.174.190:33143)
0808 04:58:32.597565 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:32.597567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:32.597699 (+ 132us) server_negotiation.cc:407] Connection header received
0808 04:58:32.597780 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:32.597781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:32.597790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:32.597826 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:32.598222 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:32.598761 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:32.599805 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:32.600030 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:32.600077 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:32.600692 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:32.600693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:32.600693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:32.600873 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:32.600892 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:32.600900 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:32.600942 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:32.601012 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 04:58:32.601036 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 04:58:32.603351 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 8819
2026-08-08T04:58:32Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1449 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:32 dist-test-slave-gjw7 krb5kdc[8990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:32 dist-test-slave-gjw7 krb5kdc[8990](info): set up 2 sockets
Aug 08 04:58:32 dist-test-slave-gjw7 krb5kdc[8990](info): commencing operation
krb5kdc: starting...
W20260808 04:58:33.874420 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260808 04:58:34.102300 3770 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.228s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260808 04:58:34.382838 3770 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.280s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260808 04:58:34.612649 3770 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.230s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/test-user.keytab.
W20260808 04:58:35.060614 3770 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.448s user 0.000s sys 0.002s
Aug 08 04:58:35 dist-test-slave-gjw7 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
W20260808 04:58:35.452165 3770 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.174.190: real 0.367s user 0.000s sys 0.003s
I20260808 04:58:35.452521 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:37741
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:46697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:37741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:35.553757 9014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:35.554040 9014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:35.554076 9014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:35.555662 9014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:35.555708 9014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:35.555721 9014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:35.555743 9014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:35.555759 9014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:35.557212 9014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:46697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:37741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:37741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:35.557467 9014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:35.557683 9014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:35.562389 9027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:35.562376 9030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:35.562489 9014 server_base.cc:1034] running on GCE node
W20260808 04:58:35.562374 9026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:35.562875 9014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:35.563177 9014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:35.564321 9014 hybrid_clock.cc:648] HybridClock initialized: now 1786165115564300 us; error 32 us; skew 500 ppm
Aug 08 04:58:35 dist-test-slave-gjw7 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165115, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:35.566778 9014 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:35.567200 9014 webserver.cc:460] Webserver started at http://127.3.174.190:43999/ using document root <none> and password file <none>
I20260808 04:58:35.567402 9014 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:35.567451 9014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:35.567575 9014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:35.568630 9014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "365d0378c17146b8a72a5b4fd5662e8b"
format_stamp: "Formatted at 2026-08-08 04:58:35 on dist-test-slave-gjw7"
server_key: "8b478def46cd130b15eaa46b62bfd193"
server_key_iv: "931d72fdd0709e5e530809d55f40c623"
server_key_version: "encryptionkey@0"
I20260808 04:58:35.568938 9014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "365d0378c17146b8a72a5b4fd5662e8b"
format_stamp: "Formatted at 2026-08-08 04:58:35 on dist-test-slave-gjw7"
server_key: "8b478def46cd130b15eaa46b62bfd193"
server_key_iv: "931d72fdd0709e5e530809d55f40c623"
server_key_version: "encryptionkey@0"
I20260808 04:58:35.570199 9014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:35.570880 9040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:35.571038 9014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:35.571125 9014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "365d0378c17146b8a72a5b4fd5662e8b"
format_stamp: "Formatted at 2026-08-08 04:58:35 on dist-test-slave-gjw7"
server_key: "8b478def46cd130b15eaa46b62bfd193"
server_key_iv: "931d72fdd0709e5e530809d55f40c623"
server_key_version: "encryptionkey@0"
I20260808 04:58:35.571189 9014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:35.595610 9014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:35.597680 9014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:35.597812 9014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:35.601531 9014 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:37741
I20260808 04:58:35.601576 9130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:37741 every 8 connection(s)
I20260808 04:58:35.601897 9014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:35.602871 9132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:35.605072 9132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b: Bootstrap starting.
I20260808 04:58:35.605648 9132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:35.605907 9132 log.cc:824] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:35.606598 9132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b: No bootstrap required, opened a new log
I20260808 04:58:35.608176 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9014
I20260808 04:58:35.608263 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:35.608157 9132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365d0378c17146b8a72a5b4fd5662e8b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37741 } }
I20260808 04:58:35.608297 9132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:35.608345 9132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365d0378c17146b8a72a5b4fd5662e8b, State: Initialized, Role: FOLLOWER
I20260808 04:58:35.608453 3770 external_mini_cluster.cc:1428] Setting key a16da7c56ce739213fc08e414895fbb9
I20260808 04:58:35.608484 9132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365d0378c17146b8a72a5b4fd5662e8b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37741 } }
I20260808 04:58:35.608566 9132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:35.608598 9132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:35.608667 9132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:35.609668 9132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365d0378c17146b8a72a5b4fd5662e8b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37741 } }
I20260808 04:58:35.609792 9132 leader_election.cc:302] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 365d0378c17146b8a72a5b4fd5662e8b; no voters:
I20260808 04:58:35.609967 9132 leader_election.cc:288] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:35.610000 9143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:35.610154 9143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [term 1 LEADER]: Becoming Leader. State: Replica: 365d0378c17146b8a72a5b4fd5662e8b, State: Running, Role: LEADER
I20260808 04:58:35.610222 9132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:35.610293 9143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365d0378c17146b8a72a5b4fd5662e8b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37741 } }
I20260808 04:58:35.610666 9143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365d0378c17146b8a72a5b4fd5662e8b. Latest consensus state: current_term: 1 leader_uuid: "365d0378c17146b8a72a5b4fd5662e8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365d0378c17146b8a72a5b4fd5662e8b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37741 } } }
I20260808 04:58:35.610877 9143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:35.610807 9145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "365d0378c17146b8a72a5b4fd5662e8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365d0378c17146b8a72a5b4fd5662e8b" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37741 } } }
I20260808 04:58:35.611267 9145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b [sys.catalog]: This master's current role is: LEADER
W20260808 04:58:35.611924 9165 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:58:35.611966 9165 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:58:35.612007 9155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:35.612151 9155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:35.613555 9155 catalog_manager.cc:1347] Generated new cluster ID: 386d968eddb840bbb042947ba115abdb
I20260808 04:58:35.613612 9155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 04:58:35 dist-test-slave-gjw7 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:35.618592 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:35.608886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47986 (local address 127.3.174.190:37741)
0808 04:58:35.609052 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:35.609060 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:35.609089 (+ 29us) server_negotiation.cc:407] Connection header received
0808 04:58:35.609191 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:35.609196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:35.609238 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:35.609375 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:35.610130 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.611519 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.612610 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.613188 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.613329 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:35.615628 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:35.615633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:35.615633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:35.615645 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:35.617155 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:35.617530 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:35.617533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:35.617533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:35.617630 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:35.617805 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:35.617805 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:35.617806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:35.618038 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:35.618216 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:35.618284 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 04:58:35.618417 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260808 04:58:35.621870 9155 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:35.622354 9155 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:35.629419 9155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 365d0378c17146b8a72a5b4fd5662e8b: Generated new TSK 0
I20260808 04:58:35.629592 9155 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:35.661201 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37741
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:46697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:35.762848 9174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:35.763052 9174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:35.763074 9174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:35.764142 9174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:35.764174 9174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:35.764199 9174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:35.765626 9174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:46697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:35.765825 9174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:35.766001 9174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:35.770737 9186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:35.770743 9188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:35.770812 9185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:35.771126 9174 server_base.cc:1034] running on GCE node
I20260808 04:58:35.771343 9174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:35.771560 9174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:35.772696 9174 hybrid_clock.cc:648] HybridClock initialized: now 1786165115772682 us; error 31 us; skew 500 ppm
Aug 08 04:58:35 dist-test-slave-gjw7 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165115, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:35.775182 9174 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:35.775622 9174 webserver.cc:460] Webserver started at http://127.3.174.129:45503/ using document root <none> and password file <none>
I20260808 04:58:35.775817 9174 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:35.775861 9174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:35.776000 9174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:35.777041 9174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "811805d39e8e405090649a902d56694d"
format_stamp: "Formatted at 2026-08-08 04:58:35 on dist-test-slave-gjw7"
server_key: "f53b6bbe104c244b54eed0993f02cff5"
server_key_iv: "3b0b4c5d375479f1a950eca874ed4b97"
server_key_version: "encryptionkey@0"
I20260808 04:58:35.777420 9174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "811805d39e8e405090649a902d56694d"
format_stamp: "Formatted at 2026-08-08 04:58:35 on dist-test-slave-gjw7"
server_key: "f53b6bbe104c244b54eed0993f02cff5"
server_key_iv: "3b0b4c5d375479f1a950eca874ed4b97"
server_key_version: "encryptionkey@0"
I20260808 04:58:35.778669 9174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:35.779364 9201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:35.779512 9174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:35.779594 9174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "811805d39e8e405090649a902d56694d"
format_stamp: "Formatted at 2026-08-08 04:58:35 on dist-test-slave-gjw7"
server_key: "f53b6bbe104c244b54eed0993f02cff5"
server_key_iv: "3b0b4c5d375479f1a950eca874ed4b97"
server_key_version: "encryptionkey@0"
I20260808 04:58:35.779663 9174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:35.794461 9174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:35.796391 9174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:35.796514 9174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:35.796756 9174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:35.797080 9174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:35.797115 9174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:35.797147 9174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:35.797170 9174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:35.802676 9174 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:42767
I20260808 04:58:35.802724 9399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:42767 every 8 connection(s)
I20260808 04:58:35.803020 9174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:58:35.806841 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9174
I20260808 04:58:35.806941 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:35.807116 3770 external_mini_cluster.cc:1428] Setting key df1141943a660e617ec4fab31528e5df
Aug 08 04:58:35 dist-test-slave-gjw7 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165115, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:35.812028 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:35.803757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:41823 (local address 127.3.174.190:37741)
0808 04:58:35.803815 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:35.803817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:35.803996 (+ 179us) server_negotiation.cc:407] Connection header received
0808 04:58:35.804472 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:35.804474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:35.804486 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:35.804534 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:35.805465 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.805969 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.806933 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.807175 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.807238 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:35.809325 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:35.809327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:35.809328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:35.809336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:35.810637 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:35.811077 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:35.811079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:35.811079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:35.811146 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:35.811352 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:35.811353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:35.811354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:35.811484 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:35.811549 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:35.811810 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 04:58:35.811905 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:58:35.812125 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:35.803701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37741 (local address 127.3.174.129:41823)
0808 04:58:35.803858 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:35.803921 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:35.804395 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:35.804598 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:35.804601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:35.805171 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:35.805386 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:35.805392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.806034 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.806035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:35.806839 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:35.806842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.807246 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.807248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:35.807368 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:35.807873 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:35.807896 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:35.809206 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:35.810731 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:35.810738 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:35.810740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:35.811003 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:35.811204 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:35.811204 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:35.811205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:35.811302 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:35.811539 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:35.811543 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:35.811680 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 04:58:35.811781 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 04:58:35.811921 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260808 04:58:35.812740 9400 heartbeater.cc:347] Connected to a master server at 127.3.174.190:37741
I20260808 04:58:35.812839 9400 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:35.813019 9400 heartbeater.cc:511] Master 127.3.174.190:37741 requested a full tablet report, sending...
I20260808 04:58:35.813583 9071 ts_manager.cc:194] Registered new tserver with Master: 811805d39e8e405090649a902d56694d (127.3.174.129:42767)
I20260808 04:58:35.814174 9071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:41823
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:35.833542 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37741
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:46697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:35.917464 9409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:35.917659 9409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:35.917689 9409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:35.918812 9409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:35.918850 9409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:35.918885 9409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:35.920296 9409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:46697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:35.920540 9409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:35.920760 9409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:35.925652 9423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:35.925702 9409 server_base.cc:1034] running on GCE node
W20260808 04:58:35.925673 9420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:35.925675 9421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:35.926112 9409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:35.926378 9409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:35.927541 9409 hybrid_clock.cc:648] HybridClock initialized: now 1786165115927524 us; error 34 us; skew 500 ppm
Aug 08 04:58:35 dist-test-slave-gjw7 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165115, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:35.929921 9409 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:35.930357 9409 webserver.cc:460] Webserver started at http://127.3.174.130:46093/ using document root <none> and password file <none>
I20260808 04:58:35.930610 9409 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:35.930670 9409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:35.930809 9409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:35.931831 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "e713196081d1406d9a8bf296a4f91260"
format_stamp: "Formatted at 2026-08-08 04:58:35 on dist-test-slave-gjw7"
server_key: "7c3f2615a9298cc1acaf63691d73fee8"
server_key_iv: "84e373e96d01c41ec2c12d45e0d1f371"
server_key_version: "encryptionkey@0"
I20260808 04:58:35.932152 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "e713196081d1406d9a8bf296a4f91260"
format_stamp: "Formatted at 2026-08-08 04:58:35 on dist-test-slave-gjw7"
server_key: "7c3f2615a9298cc1acaf63691d73fee8"
server_key_iv: "84e373e96d01c41ec2c12d45e0d1f371"
server_key_version: "encryptionkey@0"
I20260808 04:58:35.933405 9409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 04:58:35.934137 9437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:35.934312 9409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:35.934396 9409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "e713196081d1406d9a8bf296a4f91260"
format_stamp: "Formatted at 2026-08-08 04:58:35 on dist-test-slave-gjw7"
server_key: "7c3f2615a9298cc1acaf63691d73fee8"
server_key_iv: "84e373e96d01c41ec2c12d45e0d1f371"
server_key_version: "encryptionkey@0"
I20260808 04:58:35.934461 9409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:35.949887 9409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:35.951572 9409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:35.951707 9409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:35.951910 9409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:35.952210 9409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:35.952253 9409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:35.952281 9409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:35.952307 9409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:35.958156 9409 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:34533
I20260808 04:58:35.958307 9637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:34533 every 8 connection(s)
I20260808 04:58:35.958534 9409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 04:58:35 dist-test-slave-gjw7 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165115, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:35.967139 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:35.959184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:35411 (local address 127.3.174.190:37741)
0808 04:58:35.959246 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:35.959248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:35.959452 (+ 204us) server_negotiation.cc:407] Connection header received
0808 04:58:35.959928 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:35.959929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:35.959938 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:35.959990 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:35.960875 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.961426 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.962387 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.962577 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.962634 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:35.964642 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:35.964643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:35.964643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:35.964650 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:35.965957 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:35.966311 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:35.966312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:35.966312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:35.966370 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:35.966588 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:35.966589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:35.966590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:35.966723 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:35.966788 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:35.966997 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 04:58:35.967071 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 04:58:35.967255 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:35.959165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37741 (local address 127.3.174.130:35411)
0808 04:58:35.959323 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:35.959366 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:35.959845 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:35.960046 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:35.960050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:35.960598 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:35.960784 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:35.960789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.961500 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.961501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:35.962300 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:35.962302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:35.962642 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:35.962643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:35.962740 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:35.963219 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:35.963228 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:35.964554 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:35.966036 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:35.966042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:35.966044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:35.966251 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:35.966420 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:35.966420 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:35.966420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:35.966510 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:35.966786 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:35.966789 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:35.966896 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 04:58:35.966972 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 04:58:35.967085 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260808 04:58:35.967758 9638 heartbeater.cc:347] Connected to a master server at 127.3.174.190:37741
I20260808 04:58:35.967844 9638 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:35.968003 9638 heartbeater.cc:511] Master 127.3.174.190:37741 requested a full tablet report, sending...
I20260808 04:58:35.968426 9071 ts_manager.cc:194] Registered new tserver with Master: e713196081d1406d9a8bf296a4f91260 (127.3.174.130:34533)
I20260808 04:58:35.968464 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9409
I20260808 04:58:35.968561 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:35.968789 3770 external_mini_cluster.cc:1428] Setting key 56150c3f8303a6eb868549433759d4c2
I20260808 04:58:35.968909 9071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:35411
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:35.994318 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37741
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:46697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:36.077407 9645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:36.077618 9645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:36.077642 9645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:36.078828 9645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:36.078876 9645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:36.078907 9645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:36.080377 9645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:46697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:36.080601 9645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:36.080827 9645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:36.085685 9653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:36.085790 9645 server_base.cc:1034] running on GCE node
W20260808 04:58:36.085670 9661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:36.085748 9652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:36.086089 9645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:36.086323 9645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:36.087466 9645 hybrid_clock.cc:648] HybridClock initialized: now 1786165116087452 us; error 27 us; skew 500 ppm
Aug 08 04:58:36 dist-test-slave-gjw7 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:36.089903 9645 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:36.090294 9645 webserver.cc:460] Webserver started at http://127.3.174.131:46125/ using document root <none> and password file <none>
I20260808 04:58:36.090514 9645 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:36.090564 9645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:36.090691 9645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:36.091707 9645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "384e7889acf341fe879dd170d6c293f4"
format_stamp: "Formatted at 2026-08-08 04:58:36 on dist-test-slave-gjw7"
server_key: "ed15e3f4a80f514fb65d7ee5ce34c635"
server_key_iv: "acf120279354a9b3d18e1645943c8833"
server_key_version: "encryptionkey@0"
I20260808 04:58:36.092049 9645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "384e7889acf341fe879dd170d6c293f4"
format_stamp: "Formatted at 2026-08-08 04:58:36 on dist-test-slave-gjw7"
server_key: "ed15e3f4a80f514fb65d7ee5ce34c635"
server_key_iv: "acf120279354a9b3d18e1645943c8833"
server_key_version: "encryptionkey@0"
I20260808 04:58:36.093243 9645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 04:58:36.093923 9676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:36.094059 9645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:36.094136 9645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "384e7889acf341fe879dd170d6c293f4"
format_stamp: "Formatted at 2026-08-08 04:58:36 on dist-test-slave-gjw7"
server_key: "ed15e3f4a80f514fb65d7ee5ce34c635"
server_key_iv: "acf120279354a9b3d18e1645943c8833"
server_key_version: "encryptionkey@0"
I20260808 04:58:36.094200 9645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:36.104779 9645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:36.106438 9645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:36.106563 9645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:36.106767 9645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:36.107049 9645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:36.107082 9645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:36.107110 9645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:36.107133 9645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:36.112852 9645 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:38793
I20260808 04:58:36.112905 9872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:38793 every 8 connection(s)
I20260808 04:58:36.113185 9645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 04:58:36 dist-test-slave-gjw7 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:36.119241 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9645
I20260808 04:58:36.119340 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:36.119524 3770 external_mini_cluster.cc:1428] Setting key c73fc9de82257b659c7754cfe41eec1f
I20260808 04:58:36.121929 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:36.113859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:51039 (local address 127.3.174.190:37741)
0808 04:58:36.113907 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:36.113909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:36.114114 (+ 205us) server_negotiation.cc:407] Connection header received
0808 04:58:36.114611 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:36.114611 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:36.114621 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:36.114661 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:36.115574 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:36.116110 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:36.117047 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:36.117221 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:36.117268 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:36.119234 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:36.119236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:36.119236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:36.119245 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:36.120814 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:36.121158 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:36.121159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:36.121159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:36.121210 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:36.121456 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:36.121456 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:36.121457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:36.121556 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:36.121605 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:36.121805 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 04:58:36.121866 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:58:36.122123 9877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:36.113834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37741 (local address 127.3.174.131:51039)
0808 04:58:36.113995 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:36.114036 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:36.114517 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:36.114727 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:36.114731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:36.115286 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:36.115489 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:36.115494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:36.116187 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:36.116188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:36.116984 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:36.116986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:36.117278 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:36.117278 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:36.117437 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:36.117851 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:36.117859 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:36.119131 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:36.120889 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:36.120895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:36.120897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:36.121098 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:36.121259 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:36.121259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:36.121259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:36.121406 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:36.121601 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:36.121604 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:36.121710 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 04:58:36.121786 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 04:58:36.121904 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260808 04:58:36.122530 9873 heartbeater.cc:347] Connected to a master server at 127.3.174.190:37741
I20260808 04:58:36.122615 9873 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:36.122764 9873 heartbeater.cc:511] Master 127.3.174.190:37741 requested a full tablet report, sending...
I20260808 04:58:36.123085 9071 ts_manager.cc:194] Registered new tserver with Master: 384e7889acf341fe879dd170d6c293f4 (127.3.174.131:38793)
I20260808 04:58:36.123440 9071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:51039
I20260808 04:58:36.131253 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 04:58:36 dist-test-slave-gjw7 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165116, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 04:58:36 dist-test-slave-gjw7 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165116, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:36.154080 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:36.147411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.3.174.190:37741)
0808 04:58:36.147468 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:36.147469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:36.147536 (+ 67us) server_negotiation.cc:407] Connection header received
0808 04:58:36.147590 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:36.147591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:36.147600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:36.147650 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:36.148142 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:36.148614 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:36.149468 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:36.149644 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:36.149690 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:36.151818 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:36.151820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:36.151820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:36.151827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:36.152945 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:36.153260 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:36.153261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:36.153262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:36.153342 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:36.153514 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:36.153515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:36.153515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:36.153680 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:36.153774 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:36.153908 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 04:58:36.153986 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 04:58:36.154284 9071 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47988
I20260808 04:58:36.154656 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9174
I20260808 04:58:36.169279 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9409
I20260808 04:58:36.178642 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9645
I20260808 04:58:36.187402 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9014
2026-08-08T04:58:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3578 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:36 dist-test-slave-gjw7 krb5kdc[9898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:36 dist-test-slave-gjw7 krb5kdc[9898](info): set up 2 sockets
Aug 08 04:58:36 dist-test-slave-gjw7 krb5kdc[9898](info): commencing operation
krb5kdc: starting...
W20260808 04:58:37.358799 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:37 dist-test-slave-gjw7 krb5kdc[9898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:37.476372 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:34263
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:38453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:34263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:37.560504 9924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:37.560724 9924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:37.560755 9924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:37.561959 9924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:37.562003 9924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:37.562021 9924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:37.562038 9924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:37.562052 9924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:37.563589 9924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:38453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:34263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:34263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:37.563853 9924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:37.564062 9924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:37.568764 9938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:37.568781 9944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:37.568928 9924 server_base.cc:1034] running on GCE node
W20260808 04:58:37.568813 9933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:37.569271 9924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:37.569597 9924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:37.570742 9924 hybrid_clock.cc:648] HybridClock initialized: now 1786165117570725 us; error 33 us; skew 500 ppm
Aug 08 04:58:37 dist-test-slave-gjw7 krb5kdc[9898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165117, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:37.576869 9924 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:37.577437 9924 webserver.cc:460] Webserver started at http://127.3.174.190:39459/ using document root <none> and password file <none>
I20260808 04:58:37.577657 9924 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:37.577708 9924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:37.577832 9924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:37.578892 9924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "8c4473e25b654be6a6c36549f6604a9f"
format_stamp: "Formatted at 2026-08-08 04:58:37 on dist-test-slave-gjw7"
server_key: "830ce4b16ff939e0341f22add4a20202"
server_key_iv: "ba9d8365ce6d6c954386f04049089a77"
server_key_version: "encryptionkey@0"
I20260808 04:58:37.579228 9924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "8c4473e25b654be6a6c36549f6604a9f"
format_stamp: "Formatted at 2026-08-08 04:58:37 on dist-test-slave-gjw7"
server_key: "830ce4b16ff939e0341f22add4a20202"
server_key_iv: "ba9d8365ce6d6c954386f04049089a77"
server_key_version: "encryptionkey@0"
I20260808 04:58:37.580514 9924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:37.581211 9968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:37.581385 9924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:37.581475 9924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "8c4473e25b654be6a6c36549f6604a9f"
format_stamp: "Formatted at 2026-08-08 04:58:37 on dist-test-slave-gjw7"
server_key: "830ce4b16ff939e0341f22add4a20202"
server_key_iv: "ba9d8365ce6d6c954386f04049089a77"
server_key_version: "encryptionkey@0"
I20260808 04:58:37.581542 9924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:37.594214 9924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:37.595978 9924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:37.596114 9924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:37.600116 9924 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:34263
I20260808 04:58:37.600149 10062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:34263 every 8 connection(s)
I20260808 04:58:37.600728 9924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:37.601352 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9924
I20260808 04:58:37.601475 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:37.601547 10063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:37.601679 3770 external_mini_cluster.cc:1428] Setting key a926ce9b45d313ca1e350887fe882828
I20260808 04:58:37.604647 10063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f: Bootstrap starting.
I20260808 04:58:37.605458 10063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:37.605762 10063 log.cc:824] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:37.606439 10063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f: No bootstrap required, opened a new log
Aug 08 04:58:37 dist-test-slave-gjw7 krb5kdc[9898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:37.608942 10063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c4473e25b654be6a6c36549f6604a9f" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 34263 } }
I20260808 04:58:37.609113 10063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:37.609139 10063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c4473e25b654be6a6c36549f6604a9f, State: Initialized, Role: FOLLOWER
I20260808 04:58:37.609279 10063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c4473e25b654be6a6c36549f6604a9f" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 34263 } }
I20260808 04:58:37.609395 10063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:37.609438 10063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:37.609491 10063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:37.610105 10063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c4473e25b654be6a6c36549f6604a9f" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 34263 } }
I20260808 04:58:37.610224 10063 leader_election.cc:302] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c4473e25b654be6a6c36549f6604a9f; no voters:
I20260808 04:58:37.610399 10063 leader_election.cc:288] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:37.610462 10074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:37.610641 10074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [term 1 LEADER]: Becoming Leader. State: Replica: 8c4473e25b654be6a6c36549f6604a9f, State: Running, Role: LEADER
I20260808 04:58:37.610674 10063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:37.610740 10074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c4473e25b654be6a6c36549f6604a9f" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 34263 } }
I20260808 04:58:37.611119 10074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c4473e25b654be6a6c36549f6604a9f. Latest consensus state: current_term: 1 leader_uuid: "8c4473e25b654be6a6c36549f6604a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c4473e25b654be6a6c36549f6604a9f" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 34263 } } }
I20260808 04:58:37.611176 10075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c4473e25b654be6a6c36549f6604a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c4473e25b654be6a6c36549f6604a9f" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 34263 } } }
I20260808 04:58:37.611302 10075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:37.611539 10074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f [sys.catalog]: This master's current role is: LEADER
W20260808 04:58:37.612567 10101 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:58:37.612672 10101 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:58:37.612766 10090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:37.612937 10090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:37.613013 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:37.602206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.3.174.190:34263)
0808 04:58:37.602393 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:37.602402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:37.602428 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:58:37.602985 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:37.602992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:37.603043 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:37.603155 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:37.603928 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.604883 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.605931 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.606423 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.606543 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:37.609092 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:37.609097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:37.609097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:37.609110 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:37.611126 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:37.611599 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:37.611602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:37.611603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:37.611720 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:37.611985 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:37.611986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:37.611986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:37.612360 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:37.612516 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:37.612653 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 04:58:37.612800 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260808 04:58:37.614502 10090 catalog_manager.cc:1347] Generated new cluster ID: f38f34347e1d47c59fc7eb25e2caa0c6
I20260808 04:58:37.614547 10090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:37.635433 10090 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:37.636243 10090 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:37.649605 10090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c4473e25b654be6a6c36549f6604a9f: Generated new TSK 0
I20260808 04:58:37.649811 10090 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:37.683521 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:34263
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:38453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:37.766069 10115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:37.766284 10115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:37.766309 10115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:37.767460 10115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:37.767500 10115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:37.767527 10115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:37.769028 10115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:38453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:34263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:37.769270 10115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:37.769510 10115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:37.774309 10124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:37.774360 10126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:37.774446 10131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:37.774521 10115 server_base.cc:1034] running on GCE node
I20260808 04:58:37.774755 10115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:37.775000 10115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:37.776147 10115 hybrid_clock.cc:648] HybridClock initialized: now 1786165117776130 us; error 33 us; skew 500 ppm
Aug 08 04:58:37 dist-test-slave-gjw7 krb5kdc[9898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165117, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:37.778703 10115 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:37.779114 10115 webserver.cc:460] Webserver started at http://127.3.174.129:35437/ using document root <none> and password file <none>
I20260808 04:58:37.779320 10115 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:37.779361 10115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:37.779455 10115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:37.780505 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "7418e4155be8418a88572ea6606ca016"
format_stamp: "Formatted at 2026-08-08 04:58:37 on dist-test-slave-gjw7"
server_key: "c4659cfa11c050cacc18662face643be"
server_key_iv: "52ada4222beedf7aa73653ee63765acc"
server_key_version: "encryptionkey@0"
I20260808 04:58:37.780812 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "7418e4155be8418a88572ea6606ca016"
format_stamp: "Formatted at 2026-08-08 04:58:37 on dist-test-slave-gjw7"
server_key: "c4659cfa11c050cacc18662face643be"
server_key_iv: "52ada4222beedf7aa73653ee63765acc"
server_key_version: "encryptionkey@0"
I20260808 04:58:37.782101 10115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 04:58:37.782791 10145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:37.782955 10115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:37.783038 10115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "7418e4155be8418a88572ea6606ca016"
format_stamp: "Formatted at 2026-08-08 04:58:37 on dist-test-slave-gjw7"
server_key: "c4659cfa11c050cacc18662face643be"
server_key_iv: "52ada4222beedf7aa73653ee63765acc"
server_key_version: "encryptionkey@0"
I20260808 04:58:37.783097 10115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:37.794046 10115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:37.796281 10115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:37.796396 10115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:37.796588 10115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:37.796862 10115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:37.796902 10115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:37.796923 10115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:37.796940 10115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:37.802512 10115 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:41141
I20260808 04:58:37.802582 10361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:41141 every 8 connection(s)
I20260808 04:58:37.802874 10115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:58:37.808818 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 10115
Aug 08 04:58:37 dist-test-slave-gjw7 krb5kdc[9898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165117, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:37.808917 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:37.809101 3770 external_mini_cluster.cc:1428] Setting key ee4fb6d03bea7ae0e6324c0586cc6994
I20260808 04:58:37.812268 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:37.803572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:56373 (local address 127.3.174.190:34263)
0808 04:58:37.803623 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:37.803625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:37.803809 (+ 184us) server_negotiation.cc:407] Connection header received
0808 04:58:37.804792 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:37.804793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:37.804807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:37.804855 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:37.805544 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.806058 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.807030 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.807253 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.807313 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:37.809394 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:37.809396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:37.809396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:37.809405 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:37.810779 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:37.811196 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:37.811196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:37.811197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:37.811271 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:37.811586 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:37.811587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:37.811587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:37.811753 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:37.811815 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:37.812086 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 04:58:37.812178 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 04:58:37.812433 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:37.803543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:34263 (local address 127.3.174.129:56373)
0808 04:58:37.803689 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:37.803726 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:37.804675 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:37.804939 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:37.804943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:37.805201 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:37.805449 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:37.805455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.806146 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.806147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:37.806939 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:37.806942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.807326 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.807327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:37.807446 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:37.807897 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:37.807906 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:37.809310 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:37.810855 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:37.810862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:37.810864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:37.811134 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:37.811362 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:37.811363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:37.811363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:37.811496 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:37.811817 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:37.811825 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:37.811963 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 04:58:37.812054 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 04:58:37.812206 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260808 04:58:37.812927 10362 heartbeater.cc:347] Connected to a master server at 127.3.174.190:34263
I20260808 04:58:37.813025 10362 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:37.813206 10362 heartbeater.cc:511] Master 127.3.174.190:34263 requested a full tablet report, sending...
I20260808 04:58:37.813830 9995 ts_manager.cc:194] Registered new tserver with Master: 7418e4155be8418a88572ea6606ca016 (127.3.174.129:41141)
I20260808 04:58:37.814739 9995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:56373
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:37.835608 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:34263
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:38453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:37.922410 10368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:37.922616 10368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:37.922639 10368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:37.923749 10368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:37.923782 10368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:37.923805 10368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:37.925282 10368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:38453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:34263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:37.925515 10368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:37.925695 10368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:37.930410 10377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:37.930416 10375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:37.930423 10380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:37.930596 10368 server_base.cc:1034] running on GCE node
I20260808 04:58:37.930819 10368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:37.931051 10368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:37.932191 10368 hybrid_clock.cc:648] HybridClock initialized: now 1786165117932174 us; error 27 us; skew 500 ppm
Aug 08 04:58:37 dist-test-slave-gjw7 krb5kdc[9898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165117, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:37.934859 10368 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:37.935297 10368 webserver.cc:460] Webserver started at http://127.3.174.130:45309/ using document root <none> and password file <none>
I20260808 04:58:37.935515 10368 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:37.935564 10368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:37.935690 10368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:37.936715 10368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "e41e34b13f5a40e895f66d511ef2068d"
format_stamp: "Formatted at 2026-08-08 04:58:37 on dist-test-slave-gjw7"
server_key: "dd42ddbb6167fcd35714b6d47aa5b986"
server_key_iv: "c67245b03a81ac7df3896b6d51536186"
server_key_version: "encryptionkey@0"
I20260808 04:58:37.937057 10368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "e41e34b13f5a40e895f66d511ef2068d"
format_stamp: "Formatted at 2026-08-08 04:58:37 on dist-test-slave-gjw7"
server_key: "dd42ddbb6167fcd35714b6d47aa5b986"
server_key_iv: "c67245b03a81ac7df3896b6d51536186"
server_key_version: "encryptionkey@0"
I20260808 04:58:37.938437 10368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:37.939078 10392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:37.939251 10368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:37.939340 10368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "e41e34b13f5a40e895f66d511ef2068d"
format_stamp: "Formatted at 2026-08-08 04:58:37 on dist-test-slave-gjw7"
server_key: "dd42ddbb6167fcd35714b6d47aa5b986"
server_key_iv: "c67245b03a81ac7df3896b6d51536186"
server_key_version: "encryptionkey@0"
I20260808 04:58:37.939414 10368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:37.964121 10368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:37.966437 10368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:37.966584 10368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:37.966822 10368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:37.967147 10368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:37.967181 10368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:37.967213 10368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:37.967236 10368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:37.972939 10368 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:40681
I20260808 04:58:37.973008 10593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:40681 every 8 connection(s)
I20260808 04:58:37.973344 10368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 04:58:37 dist-test-slave-gjw7 krb5kdc[9898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165117, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:37.981431 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 10368
I20260808 04:58:37.981537 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:37.981709 3770 external_mini_cluster.cc:1428] Setting key f768f7914b4dd6f97d3e9cfe508f93ac
I20260808 04:58:37.982112 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:37.974092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:53529 (local address 127.3.174.190:34263)
0808 04:58:37.974178 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:37.974181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:37.974342 (+ 161us) server_negotiation.cc:407] Connection header received
0808 04:58:37.975267 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:37.975269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:37.975279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:37.975326 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:37.975935 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.976426 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.977360 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.977564 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.977614 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:37.979521 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:37.979522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:37.979522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:37.979529 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:37.980750 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:37.981122 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:37.981123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:37.981123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:37.981171 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:37.981442 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:37.981443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:37.981444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:37.981582 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:37.981651 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:37.981932 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 04:58:37.982024 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 04:58:37.982270 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:37.974062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:34263 (local address 127.3.174.130:53529)
0808 04:58:37.974217 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:37.974258 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:37.975180 (+ 922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:37.975396 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:37.975400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:37.975656 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:37.975858 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:37.975863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.976483 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.976483 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:37.977272 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:37.977275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:37.977622 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:37.977623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:37.977726 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:37.978152 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:37.978160 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:37.979439 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:37.980836 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:37.980842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:37.980844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:37.981060 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:37.981218 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:37.981219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:37.981219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:37.981374 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:37.981640 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:37.981647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:37.981798 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 04:58:37.981901 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 04:58:37.982050 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260808 04:58:37.982755 10595 heartbeater.cc:347] Connected to a master server at 127.3.174.190:34263
I20260808 04:58:37.982854 10595 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:37.983042 10595 heartbeater.cc:511] Master 127.3.174.190:34263 requested a full tablet report, sending...
I20260808 04:58:37.983424 9995 ts_manager.cc:194] Registered new tserver with Master: e41e34b13f5a40e895f66d511ef2068d (127.3.174.130:40681)
I20260808 04:58:37.983793 9995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:53529
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:38.007519 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:34263
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:38453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:38.090873 10607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:38.091086 10607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:38.091109 10607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:38.092258 10607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:38.092304 10607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:38.092330 10607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:38.093904 10607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:38453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:34263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:38.094154 10607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:38.094393 10607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:38.099279 10634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:38.099294 10627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:38.099335 10623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:38.099530 10607 server_base.cc:1034] running on GCE node
I20260808 04:58:38.099763 10607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:38.099989 10607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:38.101125 10607 hybrid_clock.cc:648] HybridClock initialized: now 1786165118101113 us; error 28 us; skew 500 ppm
Aug 08 04:58:38 dist-test-slave-gjw7 krb5kdc[9898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165118, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:38.103704 10607 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:38.104254 10607 webserver.cc:460] Webserver started at http://127.3.174.131:39609/ using document root <none> and password file <none>
I20260808 04:58:38.104523 10607 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:38.104576 10607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:38.104710 10607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:38.105862 10607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "e7d8f188bf3f493785962aefde68a2d9"
format_stamp: "Formatted at 2026-08-08 04:58:38 on dist-test-slave-gjw7"
server_key: "1df762d116d3b21a7c1b61a5353715e4"
server_key_iv: "caa721f1fe80611471eb370d79835a7e"
server_key_version: "encryptionkey@0"
I20260808 04:58:38.106220 10607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "e7d8f188bf3f493785962aefde68a2d9"
format_stamp: "Formatted at 2026-08-08 04:58:38 on dist-test-slave-gjw7"
server_key: "1df762d116d3b21a7c1b61a5353715e4"
server_key_iv: "caa721f1fe80611471eb370d79835a7e"
server_key_version: "encryptionkey@0"
I20260808 04:58:38.107573 10607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:38.108227 10655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:38.108377 10607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:38.108462 10607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "e7d8f188bf3f493785962aefde68a2d9"
format_stamp: "Formatted at 2026-08-08 04:58:38 on dist-test-slave-gjw7"
server_key: "1df762d116d3b21a7c1b61a5353715e4"
server_key_iv: "caa721f1fe80611471eb370d79835a7e"
server_key_version: "encryptionkey@0"
I20260808 04:58:38.108533 10607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:38.135398 10607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:38.137704 10607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:38.137838 10607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:38.138098 10607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:38.138430 10607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:38.138465 10607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:38.138489 10607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:38.138512 10607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:38.144517 10607 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:33431
I20260808 04:58:38.144573 10862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:33431 every 8 connection(s)
I20260808 04:58:38.144893 10607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 04:58:38 dist-test-slave-gjw7 krb5kdc[9898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165118, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:38.153349 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 10607
I20260808 04:58:38.153452 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:38.153661 3770 external_mini_cluster.cc:1428] Setting key 37dd48fb3cf9983056314b8f1f1d3fce
I20260808 04:58:38.153882 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:38.145636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:43261 (local address 127.3.174.190:34263)
0808 04:58:38.145700 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:38.145702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:38.145892 (+ 190us) server_negotiation.cc:407] Connection header received
0808 04:58:38.146823 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:38.146825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:38.146836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:38.146888 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:38.147472 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:38.147964 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:38.148877 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:38.149052 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:38.149097 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:38.151450 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:38.151451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:38.151452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:38.151458 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:38.152557 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:38.152909 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:38.152909 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:38.152910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:38.152955 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:38.153142 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:38.153142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:38.153143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:38.153286 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:38.153371 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:38.153693 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 04:58:38.153795 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:58:38.154078 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:38.145607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:34263 (local address 127.3.174.131:43261)
0808 04:58:38.145772 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:38.145813 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:38.146728 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:38.146951 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:38.146955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:38.147206 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:38.147404 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:38.147408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:38.148027 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:38.148028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:38.148815 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:38.148818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:38.149111 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:38.149111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:38.149217 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:38.149783 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:38.149794 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:38.151295 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:38.152622 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:38.152628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:38.152630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:38.152838 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:38.153000 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:38.153001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:38.153001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:38.153095 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:38.153377 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:38.153385 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:38.153556 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 04:58:38.153673 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 04:58:38.153842 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260808 04:58:38.154537 10863 heartbeater.cc:347] Connected to a master server at 127.3.174.190:34263
I20260808 04:58:38.154632 10863 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:38.154811 10863 heartbeater.cc:511] Master 127.3.174.190:34263 requested a full tablet report, sending...
I20260808 04:58:38.155155 9995 ts_manager.cc:194] Registered new tserver with Master: e7d8f188bf3f493785962aefde68a2d9 (127.3.174.131:33431)
I20260808 04:58:38.155524 9995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:43261
I20260808 04:58:38.165506 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 04:58:38 dist-test-slave-gjw7 krb5kdc[9898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:58:38 dist-test-slave-gjw7 krb5kdc[9898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:38.190336 10871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:38.181840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42248 (local address 127.3.174.129:41141)
0808 04:58:38.182023 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:38.182030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:38.182044 (+ 14us) server_negotiation.cc:407] Connection header received
0808 04:58:38.182081 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:38.182083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:38.182102 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:38.182179 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:38.182894 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:38.183732 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:38.184443 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:38.184796 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:38.184856 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:38.187435 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:38.187449 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:38.187449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:38.187458 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:38.189082 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:38.189528 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:38.189531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:38.189531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:38.189614 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:38.189779 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:38.189780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:38.189780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:38.189940 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:38.190052 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:38.190165 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 04:58:38.190240 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
Aug 08 04:58:38 dist-test-slave-gjw7 krb5kdc[9898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:38.199229 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:38.191886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.3.174.190:34263)
0808 04:58:38.191942 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:38.191944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:38.191953 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:38.192010 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:38.192011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:38.192021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:38.192061 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:38.192553 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:38.193084 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:38.193884 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:38.194089 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:38.194144 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:38.196427 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:38.196429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:38.196430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:38.196458 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:38.197965 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:38.198416 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:38.198417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:38.198417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:38.198482 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:38.198697 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:38.198698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:38.198698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:38.198875 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:38.198956 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:38.199078 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 04:58:38.199150 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 04:58:38.199592 9995 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56036
I20260808 04:58:38.199956 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 10115
I20260808 04:58:38.217450 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 10368
I20260808 04:58:38.268148 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 10607
I20260808 04:58:38.290030 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9924
2026-08-08T04:58:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2121 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:38 dist-test-slave-gjw7 krb5kdc[10879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:38 dist-test-slave-gjw7 krb5kdc[10879](info): set up 2 sockets
Aug 08 04:58:38 dist-test-slave-gjw7 krb5kdc[10879](info): commencing operation
krb5kdc: starting...
W20260808 04:58:39.528244 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:39 dist-test-slave-gjw7 krb5kdc[10879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:39.647864 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46271
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:46271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:39.729280 10895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:39.729539 10895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:39.729570 10895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:39.730726 10895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:39.730764 10895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:39.730783 10895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:39.730799 10895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:39.730815 10895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:39.732333 10895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:46271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:39.732576 10895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:39.732776 10895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:39.737479 10905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:39.737519 10903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:39.737493 10902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:39.737808 10895 server_base.cc:1034] running on GCE node
I20260808 04:58:39.738165 10895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:39.738463 10895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:39.739603 10895 hybrid_clock.cc:648] HybridClock initialized: now 1786165119739589 us; error 29 us; skew 500 ppm
Aug 08 04:58:39 dist-test-slave-gjw7 krb5kdc[10879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165119, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:39.742048 10895 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:39.742398 10895 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:39.742455 10895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:39.742585 10895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:39.743635 10895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "0fdd78fbc6d64c60a092c9588bf26c66"
format_stamp: "Formatted at 2026-08-08 04:58:39 on dist-test-slave-gjw7"
server_key: "475f48abaf9549b4d8348a7d2a8c4adb"
server_key_iv: "439307d5d7b0d5cc437b1644b2c517e7"
server_key_version: "encryptionkey@0"
I20260808 04:58:39.743978 10895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "0fdd78fbc6d64c60a092c9588bf26c66"
format_stamp: "Formatted at 2026-08-08 04:58:39 on dist-test-slave-gjw7"
server_key: "475f48abaf9549b4d8348a7d2a8c4adb"
server_key_iv: "439307d5d7b0d5cc437b1644b2c517e7"
server_key_version: "encryptionkey@0"
I20260808 04:58:39.745321 10895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:39.746055 10913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:39.746194 10895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:39.746335 10895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "0fdd78fbc6d64c60a092c9588bf26c66"
format_stamp: "Formatted at 2026-08-08 04:58:39 on dist-test-slave-gjw7"
server_key: "475f48abaf9549b4d8348a7d2a8c4adb"
server_key_iv: "439307d5d7b0d5cc437b1644b2c517e7"
server_key_version: "encryptionkey@0"
I20260808 04:58:39.746402 10895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:39.755300 10895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:39.756928 10895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:39.757057 10895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:39.760771 10895 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:46271
I20260808 04:58:39.760795 11014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:46271 every 8 connection(s)
I20260808 04:58:39.761060 10895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:39.762197 11017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:39.762522 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 10895
I20260808 04:58:39.762631 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:39.762812 3770 external_mini_cluster.cc:1428] Setting key 6d75628185bf639ef21ea05700a660f1
I20260808 04:58:39.764822 11017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66: Bootstrap starting.
I20260808 04:58:39.765437 11017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:39.765687 11017 log.cc:824] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:39.766320 11017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66: No bootstrap required, opened a new log
I20260808 04:58:39.768260 11017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fdd78fbc6d64c60a092c9588bf26c66" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46271 } }
I20260808 04:58:39.768429 11017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:39.768477 11017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fdd78fbc6d64c60a092c9588bf26c66, State: Initialized, Role: FOLLOWER
I20260808 04:58:39.768605 11017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fdd78fbc6d64c60a092c9588bf26c66" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46271 } }
I20260808 04:58:39.768685 11017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:39.768723 11017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:39.768770 11017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 04:58:39 dist-test-slave-gjw7 krb5kdc[10879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:39.769373 11017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fdd78fbc6d64c60a092c9588bf26c66" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46271 } }
I20260808 04:58:39.769477 11017 leader_election.cc:302] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fdd78fbc6d64c60a092c9588bf26c66; no voters:
I20260808 04:58:39.769631 11017 leader_election.cc:288] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:39.769672 11026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:39.769773 11026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [term 1 LEADER]: Becoming Leader. State: Replica: 0fdd78fbc6d64c60a092c9588bf26c66, State: Running, Role: LEADER
I20260808 04:58:39.769873 11026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fdd78fbc6d64c60a092c9588bf26c66" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46271 } }
I20260808 04:58:39.769901 11017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:39.770273 11031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fdd78fbc6d64c60a092c9588bf26c66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fdd78fbc6d64c60a092c9588bf26c66" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46271 } } }
I20260808 04:58:39.770370 11031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:39.770263 11032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fdd78fbc6d64c60a092c9588bf26c66. Latest consensus state: current_term: 1 leader_uuid: "0fdd78fbc6d64c60a092c9588bf26c66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fdd78fbc6d64c60a092c9588bf26c66" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46271 } } }
I20260808 04:58:39.770790 11032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:39.771111 11037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:39.771595 11037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:39.773195 11037 catalog_manager.cc:1347] Generated new cluster ID: 8f8a5fa31ee746928c2fac4363fa6ee9
I20260808 04:58:39.773242 11037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:39.773442 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:39.763332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.3.174.190:46271)
0808 04:58:39.763518 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:39.763527 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:39.763553 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:58:39.763965 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:39.763971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:39.764017 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:39.764144 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:39.764779 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:39.765785 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:39.766533 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:39.766928 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:39.767020 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:39.769457 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:39.769462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:39.769462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:39.769475 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:39.771663 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:39.772013 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:39.772016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:39.772017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:39.772124 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:39.772440 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:39.772441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:39.772441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:39.772728 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:39.772874 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:39.773099 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 04:58:39.773232 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260808 04:58:39.794524 11037 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:39.795073 11037 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:39.798684 11037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66: Generated new TSK 0
I20260808 04:58:39.798878 11037 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:39.828506 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46271
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:39.913466 11065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:39.913658 11065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:39.913681 11065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:39.914795 11065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:39.914834 11065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:39.914860 11065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:39.916386 11065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:39.916620 11065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:39.916841 11065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:39.921571 11071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:39.921581 11079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:39.921583 11073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:39.921772 11065 server_base.cc:1034] running on GCE node
I20260808 04:58:39.922106 11065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:39.922324 11065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:39.923460 11065 hybrid_clock.cc:648] HybridClock initialized: now 1786165119923443 us; error 35 us; skew 500 ppm
Aug 08 04:58:39 dist-test-slave-gjw7 krb5kdc[10879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165119, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:39.926008 11065 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:39.926338 11065 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:39.926391 11065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:39.926515 11065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:39.927542 11065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "d6ab8915ce454d56824f686d36ebb8f4"
format_stamp: "Formatted at 2026-08-08 04:58:39 on dist-test-slave-gjw7"
server_key: "da180d2337325300781f07f8070fac6e"
server_key_iv: "185d55d9ccff79e53ecdbca69c351f01"
server_key_version: "encryptionkey@0"
I20260808 04:58:39.927875 11065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "d6ab8915ce454d56824f686d36ebb8f4"
format_stamp: "Formatted at 2026-08-08 04:58:39 on dist-test-slave-gjw7"
server_key: "da180d2337325300781f07f8070fac6e"
server_key_iv: "185d55d9ccff79e53ecdbca69c351f01"
server_key_version: "encryptionkey@0"
I20260808 04:58:39.929080 11065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:39.929800 11088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:39.929947 11065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:39.930032 11065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "d6ab8915ce454d56824f686d36ebb8f4"
format_stamp: "Formatted at 2026-08-08 04:58:39 on dist-test-slave-gjw7"
server_key: "da180d2337325300781f07f8070fac6e"
server_key_iv: "185d55d9ccff79e53ecdbca69c351f01"
server_key_version: "encryptionkey@0"
I20260808 04:58:39.930096 11065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:39.946480 11065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:39.948655 11065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:39.948787 11065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:39.949023 11065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:39.949364 11065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:39.949400 11065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:39.949422 11065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:39.949441 11065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:39.955144 11065 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:45907
I20260808 04:58:39.955199 11283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:45907 every 8 connection(s)
I20260808 04:58:39.955451 11065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 04:58:39 dist-test-slave-gjw7 krb5kdc[10879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165119, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:39.963696 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 11065
I20260808 04:58:39.963819 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:39.963924 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:39.956126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:55475 (local address 127.3.174.190:46271)
0808 04:58:39.956178 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:39.956180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:39.956412 (+ 232us) server_negotiation.cc:407] Connection header received
0808 04:58:39.956867 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:39.956868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:39.956878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:39.956928 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:39.957979 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:39.958502 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:39.959429 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:39.959628 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:39.959676 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:39.961691 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:39.961692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:39.961692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:39.961699 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:39.962774 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:39.963115 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:39.963115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:39.963115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:39.963159 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:39.963346 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:39.963346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:39.963346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:39.963442 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:39.963486 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:39.963730 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 04:58:39.963823 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 04:58:39.964012 3770 external_mini_cluster.cc:1428] Setting key f03227091d18792a52352dd22d258644
I20260808 04:58:39.964042 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:39.956113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46271 (local address 127.3.174.129:55475)
0808 04:58:39.956286 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:39.956330 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:39.956791 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:39.957002 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:39.957006 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:39.957259 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:39.957887 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:39.957894 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:39.958570 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:39.958571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:39.959362 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:39.959364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:39.959683 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:39.959684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:39.959778 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:39.960189 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:39.960197 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:39.961610 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:39.962843 (+ 1233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:39.962850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:39.962851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:39.963060 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:39.963201 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:39.963202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:39.963202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:39.963303 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:39.963491 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:39.963497 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:39.963613 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 04:58:39.963692 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 04:58:39.963832 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260808 04:58:39.964499 11284 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46271
I20260808 04:58:39.964603 11284 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:39.964762 11284 heartbeater.cc:511] Master 127.3.174.190:46271 requested a full tablet report, sending...
I20260808 04:58:39.965361 10945 ts_manager.cc:194] Registered new tserver with Master: d6ab8915ce454d56824f686d36ebb8f4 (127.3.174.129:45907)
I20260808 04:58:39.965961 10945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:55475
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:39.988234 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46271
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:40.069986 11292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:40.070267 11292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:40.070308 11292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:40.071503 11292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:40.071545 11292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:40.071571 11292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:40.073081 11292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:40.073328 11292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:40.073547 11292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:40.078351 11302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:40.078379 11301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:40.078351 11305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:40.078611 11292 server_base.cc:1034] running on GCE node
I20260808 04:58:40.078830 11292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:40.079071 11292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:40.080264 11292 hybrid_clock.cc:648] HybridClock initialized: now 1786165120080250 us; error 25 us; skew 500 ppm
Aug 08 04:58:40 dist-test-slave-gjw7 krb5kdc[10879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165120, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:40.082738 11292 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:40.083081 11292 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:40.083130 11292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:40.083233 11292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:40.084209 11292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "157d2dcb290a43f08e40324d63198a8f"
format_stamp: "Formatted at 2026-08-08 04:58:40 on dist-test-slave-gjw7"
server_key: "055e74e24b1878164ac67596373a7b9e"
server_key_iv: "13e1333252299cce85eeee593aeece80"
server_key_version: "encryptionkey@0"
I20260808 04:58:40.084489 11292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "157d2dcb290a43f08e40324d63198a8f"
format_stamp: "Formatted at 2026-08-08 04:58:40 on dist-test-slave-gjw7"
server_key: "055e74e24b1878164ac67596373a7b9e"
server_key_iv: "13e1333252299cce85eeee593aeece80"
server_key_version: "encryptionkey@0"
I20260808 04:58:40.085732 11292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:40.086401 11317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:40.086553 11292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:40.086635 11292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "157d2dcb290a43f08e40324d63198a8f"
format_stamp: "Formatted at 2026-08-08 04:58:40 on dist-test-slave-gjw7"
server_key: "055e74e24b1878164ac67596373a7b9e"
server_key_iv: "13e1333252299cce85eeee593aeece80"
server_key_version: "encryptionkey@0"
I20260808 04:58:40.086700 11292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:40.101495 11292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:40.103396 11292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:40.103526 11292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:40.103730 11292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:40.104022 11292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:40.104054 11292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:40.104077 11292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:40.104100 11292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:40.110121 11292 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:38655
I20260808 04:58:40.110162 11504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:38655 every 8 connection(s)
I20260808 04:58:40.110426 11292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:58:40.113087 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 11292
I20260808 04:58:40.113188 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:40.113404 3770 external_mini_cluster.cc:1428] Setting key 2f745ec86132523c60ec5fbc1d1051b4
Aug 08 04:58:40 dist-test-slave-gjw7 krb5kdc[10879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165120, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:40.119978 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.111174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:58105 (local address 127.3.174.190:46271)
0808 04:58:40.111273 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.111275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.111424 (+ 149us) server_negotiation.cc:407] Connection header received
0808 04:58:40.111933 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.111935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.111947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.111995 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:40.112934 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.113541 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.114874 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.115059 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.115106 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.117211 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:40.117213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:40.117213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:40.117221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:40.118694 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:40.119066 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:40.119067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:40.119067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:40.119129 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:40.119381 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:40.119381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:40.119382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:40.119517 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:40.119590 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.119805 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.119885 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 04:58:40.120036 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.111139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46271 (local address 127.3.174.130:58105)
0808 04:58:40.111308 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:40.111346 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:40.111834 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:40.112077 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:40.112081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:40.112330 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:40.112818 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.112823 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.113638 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.113639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.114779 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.114783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.115128 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.115129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.115264 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.115754 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:40.115766 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:40.117120 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:40.118781 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:40.118787 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:40.118789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:40.118997 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:40.119189 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:40.119189 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:40.119189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:40.119319 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:40.119577 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:40.119580 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:40.119685 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 04:58:40.119763 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 04:58:40.119878 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260808 04:58:40.120859 11505 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46271
I20260808 04:58:40.120970 11505 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:40.121127 11505 heartbeater.cc:511] Master 127.3.174.190:46271 requested a full tablet report, sending...
I20260808 04:58:40.121573 10945 ts_manager.cc:194] Registered new tserver with Master: 157d2dcb290a43f08e40324d63198a8f (127.3.174.130:38655)
I20260808 04:58:40.121932 10945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:58105
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:40.139868 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46271
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:40.223480 11511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:40.223711 11511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:40.223745 11511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:40.225023 11511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:40.225069 11511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:40.225103 11511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:40.226648 11511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:40.226897 11511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:40.227126 11511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:40.231943 11532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:40.232081 11511 server_base.cc:1034] running on GCE node
W20260808 04:58:40.232144 11525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:40.231943 11526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:40.232394 11511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:40.232625 11511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:40.233762 11511 hybrid_clock.cc:648] HybridClock initialized: now 1786165120233742 us; error 34 us; skew 500 ppm
Aug 08 04:58:40 dist-test-slave-gjw7 krb5kdc[10879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165120, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:40.236243 11511 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:40.236518 11511 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:40.236560 11511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:40.236654 11511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:40.237738 11511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "f4d7759901d1441db7c962f4bfb7c553"
format_stamp: "Formatted at 2026-08-08 04:58:40 on dist-test-slave-gjw7"
server_key: "272d8d25485e05251be48f332869c622"
server_key_iv: "66356bcf595e70987e38caf431741c59"
server_key_version: "encryptionkey@0"
I20260808 04:58:40.238044 11511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "f4d7759901d1441db7c962f4bfb7c553"
format_stamp: "Formatted at 2026-08-08 04:58:40 on dist-test-slave-gjw7"
server_key: "272d8d25485e05251be48f332869c622"
server_key_iv: "66356bcf595e70987e38caf431741c59"
server_key_version: "encryptionkey@0"
I20260808 04:58:40.239228 11511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:40.239989 11543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:40.240139 11511 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:40.240221 11511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "f4d7759901d1441db7c962f4bfb7c553"
format_stamp: "Formatted at 2026-08-08 04:58:40 on dist-test-slave-gjw7"
server_key: "272d8d25485e05251be48f332869c622"
server_key_iv: "66356bcf595e70987e38caf431741c59"
server_key_version: "encryptionkey@0"
I20260808 04:58:40.240294 11511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:40.271106 11511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:40.273463 11511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:40.273602 11511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:40.273844 11511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:40.274164 11511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:40.274201 11511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:40.274223 11511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:40.274235 11511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:40.279672 11511 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:34153
I20260808 04:58:40.279731 11740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:34153 every 8 connection(s)
I20260808 04:58:40.279971 11511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 04:58:40.285338 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 11511
I20260808 04:58:40.285439 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:40.285640 3770 external_mini_cluster.cc:1428] Setting key 0d07a70f62742f0f31cea5190243ec08
Aug 08 04:58:40 dist-test-slave-gjw7 krb5kdc[10879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165120, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:40.288641 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.280807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:41849 (local address 127.3.174.190:46271)
0808 04:58:40.280871 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.280873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.281061 (+ 188us) server_negotiation.cc:407] Connection header received
0808 04:58:40.281647 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.281648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.281658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.281726 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:40.282612 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.283113 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.284070 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.284257 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.284305 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.286569 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:40.286570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:40.286570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:40.286576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:40.287618 (+ 1042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:40.287958 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:40.287958 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:40.287958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:40.288003 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:40.288176 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:40.288176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:40.288176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:40.288268 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:40.288313 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.288522 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.288578 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 04:58:40.288854 11745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.280779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46271 (local address 127.3.174.131:41849)
0808 04:58:40.280941 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:40.280983 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:40.281560 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:40.281787 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:40.281791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:40.282039 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:40.282543 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.282549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.283183 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.283184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.284001 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.284004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.284320 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.284320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.284421 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.284862 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:40.284870 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:40.286440 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:40.287683 (+ 1243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:40.287689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:40.287691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:40.287902 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:40.288046 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:40.288046 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:40.288046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:40.288135 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:40.288312 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:40.288317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:40.288426 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 04:58:40.288506 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 04:58:40.288620 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260808 04:58:40.289229 11741 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46271
I20260808 04:58:40.289340 11741 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:40.289475 11741 heartbeater.cc:511] Master 127.3.174.190:46271 requested a full tablet report, sending...
I20260808 04:58:40.289808 10945 ts_manager.cc:194] Registered new tserver with Master: f4d7759901d1441db7c962f4bfb7c553 (127.3.174.131:34153)
I20260808 04:58:40.290150 10945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:41849
I20260808 04:58:40.297192 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:58:40.304140 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.298455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.3.174.190:46271)
0808 04:58:40.298506 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.298508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.298629 (+ 121us) server_negotiation.cc:407] Connection header received
0808 04:58:40.298669 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.298670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.298680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.298746 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:40.299268 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.299731 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.300472 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.300650 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.300702 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.301904 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:40.301906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:40.301906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:40.301913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:40.303090 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:40.303411 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:40.303413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:40.303413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:40.303463 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:40.303618 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:40.303619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:40.303619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:40.303768 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:40.303858 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.303978 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.304053 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 04:58:40.305510 10945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:40.307138 10945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:40.313237 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.309391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:34153 (local address 127.0.0.1:55828)
0808 04:58:40.309780 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:40.309787 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:40.309823 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:40.309984 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:40.309988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:40.309990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:40.310081 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.310083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.311020 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.311021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.312153 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.312155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.312951 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.312952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.313025 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.313040 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 04:58:40.313100 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 04:58:40.313156 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260808 04:58:40.313236 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.309623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55828 (local address 127.3.174.131:34153)
0808 04:58:40.309823 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.309829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.309840 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:58:40.309879 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.309881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.309896 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.309966 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:40.310166 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.310927 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.312245 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.312879 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.312938 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.312997 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.313121 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.313156 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260808 04:58:40.313500 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.309387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:38655 (local address 127.0.0.1:48774)
0808 04:58:40.309553 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:40.309593 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:40.309659 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:40.310189 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:40.310190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:40.310193 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:40.310254 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.310255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.311528 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.311529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.312400 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.312403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.313265 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.313266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.313358 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.313363 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:40.313393 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 04:58:40.313432 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":2,"thread_start_us":59,"threads_started":1}
I20260808 04:58:40.313768 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.309391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:45907 (local address 127.0.0.1:32926)
0808 04:58:40.309635 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:40.309645 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:40.309699 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:40.310096 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:40.310097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:40.310099 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:40.310152 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.310153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.311462 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.311463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.312549 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.312551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.313488 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.313489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.313543 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.313547 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:40.313653 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 04:58:40.313692 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":29,"thread_start_us":85,"threads_started":1}
I20260808 04:58:40.313823 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.309486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48774 (local address 127.3.174.130:38655)
0808 04:58:40.309923 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.309926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.309934 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:40.309971 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.309972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.309986 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.310086 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:40.310358 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.311410 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.312472 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.313148 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.313615 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.313682 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.313707 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.313748 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":81,"threads_started":1}
I20260808 04:58:40.314217 11632 tablet_service.cc:1467] Processing CreateTablet for tablet 7971683e2e9c455491b4915c97a28251 (DEFAULT_TABLE table=test-table [id=b3da314406b346969757e32b0119f47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:40.314333 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.309695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32926 (local address 127.3.174.129:45907)
0808 04:58:40.309861 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.309866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.309876 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:40.309916 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.309918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.309934 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.310010 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:40.310230 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.311359 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.312626 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.313346 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.313406 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.313456 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.314201 (+ 745us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.314247 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260808 04:58:40.314558 11394 tablet_service.cc:1467] Processing CreateTablet for tablet 7971683e2e9c455491b4915c97a28251 (DEFAULT_TABLE table=test-table [id=b3da314406b346969757e32b0119f47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:40.314667 11632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7971683e2e9c455491b4915c97a28251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:40.314900 11394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7971683e2e9c455491b4915c97a28251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:40.314910 11161 tablet_service.cc:1467] Processing CreateTablet for tablet 7971683e2e9c455491b4915c97a28251 (DEFAULT_TABLE table=test-table [id=b3da314406b346969757e32b0119f47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:40.315214 11161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7971683e2e9c455491b4915c97a28251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:40.317457 11773 tablet_bootstrap.cc:492] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: Bootstrap starting.
I20260808 04:58:40.318045 11773 tablet_bootstrap.cc:654] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:40.318169 11774 tablet_bootstrap.cc:492] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: Bootstrap starting.
I20260808 04:58:40.318833 11773 log.cc:824] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:40.318871 11774 tablet_bootstrap.cc:654] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:40.319204 11775 tablet_bootstrap.cc:492] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4: Bootstrap starting.
I20260808 04:58:40.319461 11774 log.cc:824] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:40.319568 11773 tablet_bootstrap.cc:492] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: No bootstrap required, opened a new log
I20260808 04:58:40.319641 11773 ts_tablet_manager.cc:1397] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:58:40.319856 11775 tablet_bootstrap.cc:654] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:40.320355 11774 tablet_bootstrap.cc:492] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: No bootstrap required, opened a new log
I20260808 04:58:40.320397 11775 log.cc:824] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:40.320427 11774 ts_tablet_manager.cc:1397] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:58:40.321272 11775 tablet_bootstrap.cc:492] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4: No bootstrap required, opened a new log
I20260808 04:58:40.321368 11775 ts_tablet_manager.cc:1397] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:40.321487 11773 raft_consensus.cc:348] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } }
I20260808 04:58:40.321653 11773 raft_consensus.cc:374] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:40.321735 11773 raft_consensus.cc:729] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d7759901d1441db7c962f4bfb7c553, State: Initialized, Role: FOLLOWER
I20260808 04:58:40.321871 11773 consensus_queue.cc:260] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } }
I20260808 04:58:40.322156 11773 ts_tablet_manager.cc:1428] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 04:58:40.322162 11741 heartbeater.cc:503] Master 127.3.174.190:46271 was elected leader, sending a full tablet report...
I20260808 04:58:40.322134 11774 raft_consensus.cc:348] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } }
I20260808 04:58:40.322301 11774 raft_consensus.cc:374] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:40.322335 11774 raft_consensus.cc:729] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157d2dcb290a43f08e40324d63198a8f, State: Initialized, Role: FOLLOWER
I20260808 04:58:40.322440 11774 consensus_queue.cc:260] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } }
I20260808 04:58:40.322751 11774 ts_tablet_manager.cc:1428] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:58:40.322767 11505 heartbeater.cc:503] Master 127.3.174.190:46271 was elected leader, sending a full tablet report...
I20260808 04:58:40.323225 11775 raft_consensus.cc:348] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } }
I20260808 04:58:40.323415 11775 raft_consensus.cc:374] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:40.323472 11775 raft_consensus.cc:729] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ab8915ce454d56824f686d36ebb8f4, State: Initialized, Role: FOLLOWER
I20260808 04:58:40.323567 11775 consensus_queue.cc:260] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } }
I20260808 04:58:40.323841 11775 ts_tablet_manager.cc:1428] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 04:58:40.323848 11284 heartbeater.cc:503] Master 127.3.174.190:46271 was elected leader, sending a full tablet report...
W20260808 04:58:40.360937 11507 tablet.cc:2369] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:58:40.456076 11286 tablet.cc:2369] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:58:40.530510 11743 tablet.cc:2369] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:40.630789 11784 raft_consensus.cc:482] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:40.630918 11784 raft_consensus.cc:504] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } }
I20260808 04:58:40.631327 11784 leader_election.cc:288] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 157d2dcb290a43f08e40324d63198a8f (127.3.174.130:38655), d6ab8915ce454d56824f686d36ebb8f4 (127.3.174.129:45907)
I20260808 04:58:40.634486 11745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.631572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:38655 (local address 127.3.174.131:55563)
0808 04:58:40.631622 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:40.631635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:40.631684 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:40.631874 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:40.631875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:40.631879 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:40.631956 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.631959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.632701 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.632702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.633636 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.633639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.634228 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.634229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.634316 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.634320 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:40.634382 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 04:58:40.634423 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":24}
I20260808 04:58:40.634491 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.631617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:55563 (local address 127.3.174.130:38655)
0808 04:58:40.631659 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.631661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.631674 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:58:40.631768 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.631769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.631779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.631848 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:40.632041 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.632625 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.633743 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.634155 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.634208 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.634216 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.634387 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.634421 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 04:58:40.634639 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.631572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:45907 (local address 127.3.174.131:36415)
0808 04:58:40.631944 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:40.631954 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:40.632005 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:40.632112 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:40.632113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:40.632115 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:40.632177 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.632179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.632871 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.632871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.633790 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:40.633793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.634394 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.634395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:40.634467 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.634471 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:40.634528 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 04:58:40.634563 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":176,"thread_start_us":124,"threads_started":1}
I20260808 04:58:40.634645 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.631711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:36415 (local address 127.3.174.129:45907)
0808 04:58:40.631772 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.631774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.632018 (+ 244us) server_negotiation.cc:407] Connection header received
0808 04:58:40.632053 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.632054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.632063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.632109 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:40.632239 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.632758 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.633880 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.634312 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.634391 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.634399 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.634547 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.634583 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 04:58:40.634859 11431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7971683e2e9c455491b4915c97a28251" candidate_uuid: "f4d7759901d1441db7c962f4bfb7c553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157d2dcb290a43f08e40324d63198a8f" is_pre_election: true
I20260808 04:58:40.635030 11431 raft_consensus.cc:2454] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4d7759901d1441db7c962f4bfb7c553 in term 0.
I20260808 04:58:40.635015 11201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7971683e2e9c455491b4915c97a28251" candidate_uuid: "f4d7759901d1441db7c962f4bfb7c553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6ab8915ce454d56824f686d36ebb8f4" is_pre_election: true
I20260808 04:58:40.635195 11201 raft_consensus.cc:2454] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4d7759901d1441db7c962f4bfb7c553 in term 0.
I20260808 04:58:40.635277 11546 leader_election.cc:302] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 157d2dcb290a43f08e40324d63198a8f, f4d7759901d1441db7c962f4bfb7c553; no voters:
I20260808 04:58:40.635435 11784 raft_consensus.cc:2781] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:40.635496 11784 raft_consensus.cc:482] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:40.635519 11784 raft_consensus.cc:3037] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:40.636272 11784 raft_consensus.cc:504] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } }
I20260808 04:58:40.636399 11784 leader_election.cc:288] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [CANDIDATE]: Term 1 election: Requested vote from peers 157d2dcb290a43f08e40324d63198a8f (127.3.174.130:38655), d6ab8915ce454d56824f686d36ebb8f4 (127.3.174.129:45907)
I20260808 04:58:40.636570 11431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7971683e2e9c455491b4915c97a28251" candidate_uuid: "f4d7759901d1441db7c962f4bfb7c553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157d2dcb290a43f08e40324d63198a8f"
I20260808 04:58:40.636595 11201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7971683e2e9c455491b4915c97a28251" candidate_uuid: "f4d7759901d1441db7c962f4bfb7c553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6ab8915ce454d56824f686d36ebb8f4"
I20260808 04:58:40.636659 11201 raft_consensus.cc:3037] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:40.636662 11431 raft_consensus.cc:3037] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:40.637348 11201 raft_consensus.cc:2454] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4d7759901d1441db7c962f4bfb7c553 in term 1.
I20260808 04:58:40.637509 11547 leader_election.cc:302] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6ab8915ce454d56824f686d36ebb8f4, f4d7759901d1441db7c962f4bfb7c553; no voters:
I20260808 04:58:40.637571 11431 raft_consensus.cc:2454] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4d7759901d1441db7c962f4bfb7c553 in term 1.
I20260808 04:58:40.637599 11784 raft_consensus.cc:2781] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:40.637746 11784 raft_consensus.cc:686] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 1 LEADER]: Becoming Leader. State: Replica: f4d7759901d1441db7c962f4bfb7c553, State: Running, Role: LEADER
I20260808 04:58:40.637842 11784 consensus_queue.cc:237] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } }
I20260808 04:58:40.638649 10946 catalog_manager.cc:5495] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4d7759901d1441db7c962f4bfb7c553 (127.3.174.131). New cstate: current_term: 1 leader_uuid: "f4d7759901d1441db7c962f4bfb7c553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d7759901d1441db7c962f4bfb7c553" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 } health_report { overall_health: UNKNOWN } } }
I20260808 04:58:40.664281 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.662046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55834 (local address 127.3.174.131:34153)
0808 04:58:40.662105 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.662107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.662115 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:40.662155 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.662156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.662182 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.662253 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:40.662377 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.662861 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.663560 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:40.663768 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:40.663824 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:40.663977 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:40.664086 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:40.664157 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.664169 (+ 12us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.664187 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:58:40.666129 11431 raft_consensus.cc:1261] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [term 1 FOLLOWER]: Refusing update from remote peer f4d7759901d1441db7c962f4bfb7c553: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:40.666142 11201 raft_consensus.cc:1261] T 7971683e2e9c455491b4915c97a28251 P d6ab8915ce454d56824f686d36ebb8f4 [term 1 FOLLOWER]: Refusing update from remote peer f4d7759901d1441db7c962f4bfb7c553: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:40.666477 11792 consensus_queue.cc:1035] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "157d2dcb290a43f08e40324d63198a8f" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:40.666602 11784 consensus_queue.cc:1035] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6ab8915ce454d56824f686d36ebb8f4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:40.668967 11798 mvcc.cc:205] Tried to move back new op lower bound from 7316132334245806080 to 7316132334133035008. Current Snapshot: MvccSnapshot[applied={T|T < 7316132334245806080}]
I20260808 04:58:40.669682 11802 mvcc.cc:205] Tried to move back new op lower bound from 7316132334245806080 to 7316132334133035008. Current Snapshot: MvccSnapshot[applied={T|T < 7316132334245806080}]
I20260808 04:58:40.671289 10945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:40.671375 10945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:40.672055 10945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fdd78fbc6d64c60a092c9588bf26c66: Sending DeleteTablet for 3 replicas of tablet 7971683e2e9c455491b4915c97a28251
I20260808 04:58:40.672377 11394 tablet_service.cc:1514] Processing DeleteTablet for tablet 7971683e2e9c455491b4915c97a28251 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:40 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:48774
I20260808 04:58:40.672585 11632 tablet_service.cc:1514] Processing DeleteTablet for tablet 7971683e2e9c455491b4915c97a28251 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:40 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:55828
I20260808 04:58:40.672608 11816 tablet_replica.cc:330] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: stopping tablet replica
I20260808 04:58:40.672722 11816 raft_consensus.cc:2229] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:40.672773 11817 tablet_replica.cc:330] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: stopping tablet replica
I20260808 04:58:40.672827 11816 raft_consensus.cc:2258] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:40.672861 11817 raft_consensus.cc:2229] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:40.672981 11817 raft_consensus.cc:2258] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:40.673007 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 11065
I20260808 04:58:40.676445 11816 ts_tablet_manager.cc:1905] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:40.678143 11816 ts_tablet_manager.cc:1918] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:40.678212 11816 log.cc:1196] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/7971683e2e9c455491b4915c97a28251
I20260808 04:58:40.678458 11816 ts_tablet_manager.cc:1939] T 7971683e2e9c455491b4915c97a28251 P 157d2dcb290a43f08e40324d63198a8f: Deleting consensus metadata
I20260808 04:58:40.678948 10917 catalog_manager.cc:4841] TS 157d2dcb290a43f08e40324d63198a8f (127.3.174.130:38655): tablet 7971683e2e9c455491b4915c97a28251 (table test-table [id=b3da314406b346969757e32b0119f47f]) successfully deleted
I20260808 04:58:40.709954 11817 ts_tablet_manager.cc:1905] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:40.712100 11817 ts_tablet_manager.cc:1918] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:40.712191 11817 log.cc:1196] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/7971683e2e9c455491b4915c97a28251
I20260808 04:58:40.712446 11817 ts_tablet_manager.cc:1939] T 7971683e2e9c455491b4915c97a28251 P f4d7759901d1441db7c962f4bfb7c553: Deleting consensus metadata
I20260808 04:58:40.712930 10915 catalog_manager.cc:4841] TS f4d7759901d1441db7c962f4bfb7c553 (127.3.174.131:34153): tablet 7971683e2e9c455491b4915c97a28251 (table test-table [id=b3da314406b346969757e32b0119f47f]) successfully deleted
W20260808 04:58:40.714926 10918 connection.cc:535] client connection to 127.3.174.129:45907 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 04:58:40.715054 10918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 04:58:40.715222 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 11292
I20260808 04:58:40.722189 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.722067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:45907 (local address 127.0.0.1:32932)
0808 04:58:40.722117 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:40.722151 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 04:58:40.722304 10918 catalog_manager.cc:4584] TS d6ab8915ce454d56824f686d36ebb8f4 (127.3.174.129:45907): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7971683e2e9c455491b4915c97a28251: Network error: Client connection negotiation failed: client connection to 127.3.174.129:45907: connect: Connection refused (error 111)
I20260808 04:58:40.732096 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 11511
I20260808 04:58:40.740839 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 10895
2026-08-08T04:58:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2434 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T04:58:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:40Z Disabled control of system clock
I20260808 04:58:40.775691 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46487
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:46487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 04:58:40.857849 11822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:40.858048 11822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:40.858068 11822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:40.859128 11822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:40.859159 11822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:40.859171 11822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:40.859181 11822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:40.859191 11822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:40.860545 11822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:46487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:40.860749 11822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:40.860972 11822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:40.865873 11833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:40.866034 11822 server_base.cc:1034] running on GCE node
W20260808 04:58:40.865882 11837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:40.865882 11832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:40.866395 11822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:40.866659 11822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:40.867785 11822 hybrid_clock.cc:648] HybridClock initialized: now 1786165120867770 us; error 32 us; skew 500 ppm
I20260808 04:58:40.869012 11822 webserver.cc:460] Webserver started at http://127.3.174.190:33809/ using document root <none> and password file <none>
I20260808 04:58:40.869239 11822 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:40.869305 11822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:40.869441 11822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:40.870560 11822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36"
format_stamp: "Formatted at 2026-08-08 04:58:40 on dist-test-slave-gjw7"
server_key: "dda93970e85d95241a1edd250e24c2de"
server_key_iv: "08cfc6b1047b550000ba1e8ccfe04047"
server_key_version: "encryptionkey@0"
I20260808 04:58:40.870896 11822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36"
format_stamp: "Formatted at 2026-08-08 04:58:40 on dist-test-slave-gjw7"
server_key: "dda93970e85d95241a1edd250e24c2de"
server_key_iv: "08cfc6b1047b550000ba1e8ccfe04047"
server_key_version: "encryptionkey@0"
I20260808 04:58:40.872139 11822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:40.872826 11845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:40.872963 11822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:40.873050 11822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36"
format_stamp: "Formatted at 2026-08-08 04:58:40 on dist-test-slave-gjw7"
server_key: "dda93970e85d95241a1edd250e24c2de"
server_key_iv: "08cfc6b1047b550000ba1e8ccfe04047"
server_key_version: "encryptionkey@0"
I20260808 04:58:40.873114 11822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:40.876449 11822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:40.876700 11822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:40.876819 11822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:40.880630 11822 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:46487
I20260808 04:58:40.880682 11940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:46487 every 8 connection(s)
I20260808 04:58:40.880975 11822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:40.881664 11942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:40.884423 11942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36: Bootstrap starting.
I20260808 04:58:40.885012 11942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:40.885252 11942 log.cc:824] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:40.885902 11942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36: No bootstrap required, opened a new log
I20260808 04:58:40.887421 11942 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46487 } }
I20260808 04:58:40.887553 11942 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:40.887574 11942 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b9c3a5f8dc45ceb8413d79bb9c5b36, State: Initialized, Role: FOLLOWER
I20260808 04:58:40.887686 11942 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46487 } }
I20260808 04:58:40.887768 11942 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:40.887795 11942 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:40.887828 11942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:40.888284 11942 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46487 } }
I20260808 04:58:40.888371 11942 leader_election.cc:302] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6b9c3a5f8dc45ceb8413d79bb9c5b36; no voters:
I20260808 04:58:40.888494 11942 leader_election.cc:288] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:40.888554 11950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:40.888706 11950 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [term 1 LEADER]: Becoming Leader. State: Replica: b6b9c3a5f8dc45ceb8413d79bb9c5b36, State: Running, Role: LEADER
I20260808 04:58:40.888728 11942 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:40.888866 11950 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46487 } }
I20260808 04:58:40.889247 11952 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6b9c3a5f8dc45ceb8413d79bb9c5b36. Latest consensus state: current_term: 1 leader_uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46487 } } }
I20260808 04:58:40.889351 11952 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:40.889469 11951 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b9c3a5f8dc45ceb8413d79bb9c5b36" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46487 } } }
I20260808 04:58:40.889523 11951 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:40.889825 11962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:40.890201 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 11822
I20260808 04:58:40.890312 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:40.890388 11962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:40.890506 3770 external_mini_cluster.cc:1428] Setting key f783135ac277bf0e3034f70f240ee8f4
I20260808 04:58:40.892091 11962 catalog_manager.cc:1347] Generated new cluster ID: 94d8aa83ab3a42479b227eb79dd28e33
I20260808 04:58:40.892143 11962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:40.892772 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:40.890961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.3.174.190:46487)
0808 04:58:40.891306 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:40.891314 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:40.891338 (+ 24us) server_negotiation.cc:407] Connection header received
0808 04:58:40.891437 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:40.891442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:40.891476 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:40.891607 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:40.892261 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:40.892264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:40.892265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:40.892268 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:40.892310 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:40.892323 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:40.892386 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:40.892459 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 04:58:40.892595 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":76,"threads_started":1}
I20260808 04:58:40.898248 11962 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:40.898404 11962 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:40.905561 11962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36: Generated new TSK 0
I20260808 04:58:40.905768 11962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:58:40.910995 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46487
--builtin_ntp_servers=127.3.174.148:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 04:58:40.994457 11989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:40.994655 11989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:40.994685 11989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:40.995810 11989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:40.995848 11989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:40.995882 11989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:40.997224 11989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:40.997471 11989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:40.997745 11989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:41.003291 11999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:41.003270 12004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:41.003400 11989 server_base.cc:1034] running on GCE node
W20260808 04:58:41.003266 11998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:41.003667 11989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:41.003903 11989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:41.005052 11989 hybrid_clock.cc:648] HybridClock initialized: now 1786165121005041 us; error 29 us; skew 500 ppm
I20260808 04:58:41.006337 11989 webserver.cc:460] Webserver started at http://127.3.174.129:38873/ using document root <none> and password file <none>
I20260808 04:58:41.006567 11989 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:41.006618 11989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:41.006762 11989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:41.007798 11989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "abc763c858904e398fae375c119e21ca"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "7acba3b0fe6bcd3bd6fcfcbfd23f0f88"
server_key_iv: "f7f7ccd7a85b3868325ddcda533c924d"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.008138 11989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "abc763c858904e398fae375c119e21ca"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "7acba3b0fe6bcd3bd6fcfcbfd23f0f88"
server_key_iv: "f7f7ccd7a85b3868325ddcda533c924d"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.009464 11989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:41.010213 12013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.010375 11989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.010468 11989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "abc763c858904e398fae375c119e21ca"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "7acba3b0fe6bcd3bd6fcfcbfd23f0f88"
server_key_iv: "f7f7ccd7a85b3868325ddcda533c924d"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.010536 11989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:41.013752 11989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:41.013993 11989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:41.014108 11989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:41.014295 11989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:41.014596 11989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:41.014628 11989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.014654 11989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:41.014679 11989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.020170 11989 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:32909
I20260808 04:58:41.020233 12210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:32909 every 8 connection(s)
I20260808 04:58:41.020629 11989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:58:41.023442 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.021363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:59677 (local address 127.3.174.190:46487)
0808 04:58:41.021413 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.021415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.021640 (+ 225us) server_negotiation.cc:407] Connection header received
0808 04:58:41.022103 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.022104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.022113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.022164 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.023213 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.023215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.023215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.023221 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.023239 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.023243 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.023292 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.023358 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.023374 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 04:58:41.023618 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.021354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46487 (local address 127.3.174.129:59677)
0808 04:58:41.021530 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.021563 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.022010 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.022242 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.022246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.022257 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:41.023070 (+ 813us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:58:41.023070 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:41.023097 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:58:41.023126 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:41.023291 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:41.023297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:41.023300 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:41.023369 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 04:58:41.023430 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260808 04:58:41.024055 12212 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46487
I20260808 04:58:41.024148 12212 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:41.024329 12212 heartbeater.cc:511] Master 127.3.174.190:46487 requested a full tablet report, sending...
I20260808 04:58:41.024802 11872 ts_manager.cc:194] Registered new tserver with Master: abc763c858904e398fae375c119e21ca (127.3.174.129:32909)
I20260808 04:58:41.026818 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 11989
I20260808 04:58:41.026906 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:41.027071 3770 external_mini_cluster.cc:1428] Setting key 50e1899ad441e711fcd6d695f81525a2
I20260808 04:58:41.028060 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46487
--builtin_ntp_servers=127.3.174.148:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 04:58:41.110672 12217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:41.110885 12217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:41.110914 12217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:41.112061 12217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:41.112098 12217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:41.112130 12217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:41.113515 12217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:41.113811 12217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:41.114135 12217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:58:41.119117 12217 server_base.cc:1034] running on GCE node
W20260808 04:58:41.119098 12228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:41.119100 12224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:41.119159 12225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:41.119449 12217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:41.119675 12217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:41.120822 12217 hybrid_clock.cc:648] HybridClock initialized: now 1786165121120804 us; error 32 us; skew 500 ppm
I20260808 04:58:41.122081 12217 webserver.cc:460] Webserver started at http://127.3.174.130:46001/ using document root <none> and password file <none>
I20260808 04:58:41.122315 12217 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:41.122368 12217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:41.122504 12217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:41.123680 12217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "6393056859f44d0391e45e9faf3d63b1"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "1fe8e44b52c6ba134e2561c02ad7416b"
server_key_iv: "7bed485354f5f498b421a86d4d883817"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.124099 12217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "6393056859f44d0391e45e9faf3d63b1"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "1fe8e44b52c6ba134e2561c02ad7416b"
server_key_iv: "7bed485354f5f498b421a86d4d883817"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.125499 12217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:41.126237 12234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.126400 12217 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:41.126502 12217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "6393056859f44d0391e45e9faf3d63b1"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "1fe8e44b52c6ba134e2561c02ad7416b"
server_key_iv: "7bed485354f5f498b421a86d4d883817"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.126582 12217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:41.130019 12217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:41.130280 12217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:41.130414 12217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:41.130613 12217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:41.130946 12217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:41.130985 12217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.131016 12217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:41.131040 12217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.136976 12217 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:37683
I20260808 04:58:41.137013 12416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:37683 every 8 connection(s)
I20260808 04:58:41.137388 12217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:58:41.140220 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.138151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:51577 (local address 127.3.174.190:46487)
0808 04:58:41.138225 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.138227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.138377 (+ 150us) server_negotiation.cc:407] Connection header received
0808 04:58:41.138857 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.138859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.138865 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.138929 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.140047 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.140048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.140048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.140050 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.140061 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.140064 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.140109 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.140155 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.140166 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 04:58:41.140391 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.138067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46487 (local address 127.3.174.130:51577)
0808 04:58:41.138251 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.138292 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.138742 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.138984 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.138988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.138997 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:41.139928 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:58:41.139928 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:41.139957 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:58:41.139986 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:41.140111 (+ 125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:41.140113 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:41.140116 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:41.140162 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 04:58:41.140210 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260808 04:58:41.140801 12419 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46487
I20260808 04:58:41.140893 12419 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:41.141037 12419 heartbeater.cc:511] Master 127.3.174.190:46487 requested a full tablet report, sending...
I20260808 04:58:41.141376 11872 ts_manager.cc:194] Registered new tserver with Master: 6393056859f44d0391e45e9faf3d63b1 (127.3.174.130:37683)
I20260808 04:58:41.143573 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 12217
I20260808 04:58:41.143666 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:41.143831 3770 external_mini_cluster.cc:1428] Setting key 35c2ce6178ec9039640f4bea00fd6b41
I20260808 04:58:41.144657 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46487
--builtin_ntp_servers=127.3.174.148:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 04:58:41.229435 12425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:41.229645 12425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:41.229673 12425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:41.230870 12425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:41.230909 12425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:41.230943 12425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:41.232277 12425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:41.232512 12425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:41.232798 12425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:41.237589 12438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:41.237603 12434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:41.237807 12425 server_base.cc:1034] running on GCE node
W20260808 04:58:41.237589 12433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:41.238041 12425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:41.238320 12425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:41.239466 12425 hybrid_clock.cc:648] HybridClock initialized: now 1786165121239451 us; error 29 us; skew 500 ppm
I20260808 04:58:41.240867 12425 webserver.cc:460] Webserver started at http://127.3.174.131:32789/ using document root <none> and password file <none>
I20260808 04:58:41.241117 12425 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:41.241184 12425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:41.241358 12425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:41.242496 12425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "80524c6f6297426ba7abf7b68b4a9bdc"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "1afc0e60011684175711bc1aaeed28b7"
server_key_iv: "3debb77462fa560c458d400fbf01dd27"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.242867 12425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "80524c6f6297426ba7abf7b68b4a9bdc"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "1afc0e60011684175711bc1aaeed28b7"
server_key_iv: "3debb77462fa560c458d400fbf01dd27"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.244285 12425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:41.245003 12447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.245162 12425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:41.245252 12425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "80524c6f6297426ba7abf7b68b4a9bdc"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "1afc0e60011684175711bc1aaeed28b7"
server_key_iv: "3debb77462fa560c458d400fbf01dd27"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.245357 12425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:41.248816 12425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:41.249089 12425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:41.249233 12425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:41.249454 12425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:41.249765 12425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:41.249804 12425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.249835 12425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:41.249863 12425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.255895 12425 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:40091
I20260808 04:58:41.255941 12658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:40091 every 8 connection(s)
I20260808 04:58:41.256278 12425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 04:58:41.259176 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.257024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:37481 (local address 127.3.174.190:46487)
0808 04:58:41.257067 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.257069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.257335 (+ 266us) server_negotiation.cc:407] Connection header received
0808 04:58:41.257815 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.257817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.257824 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.257876 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.258983 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.258984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.258984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.258988 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.259001 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.259004 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.259055 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.259101 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.259114 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 04:58:41.259351 12664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.257002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46487 (local address 127.3.174.131:37481)
0808 04:58:41.257161 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.257194 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.257718 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.257938 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.257942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.257952 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:41.258844 (+ 892us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:58:41.258845 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:41.258875 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:58:41.258907 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:41.259053 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:41.259055 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:41.259058 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:41.259105 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 04:58:41.259153 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260808 04:58:41.259840 12661 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46487
I20260808 04:58:41.259917 12661 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:41.260061 12661 heartbeater.cc:511] Master 127.3.174.190:46487 requested a full tablet report, sending...
I20260808 04:58:41.260239 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 12425
I20260808 04:58:41.260340 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:41.260455 11872 ts_manager.cc:194] Registered new tserver with Master: 80524c6f6297426ba7abf7b68b4a9bdc (127.3.174.131:40091)
I20260808 04:58:41.260509 3770 external_mini_cluster.cc:1428] Setting key 30d6244a2b3cae3d7d3b963084c7029d
I20260808 04:58:41.261277 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:58:41.263494 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.262546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.3.174.190:46487)
0808 04:58:41.262637 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.262639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.262693 (+ 54us) server_negotiation.cc:407] Connection header received
0808 04:58:41.262767 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.262768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.262772 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.262819 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.263289 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.263290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.263290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.263294 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.263327 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.263331 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.263384 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.263416 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.263428 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 04:58:41.264856 11872 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:41.266393 11872 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:41.269968 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.268458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:40091 (local address 127.0.0.1:48300)
0808 04:58:41.268626 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.268663 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.268715 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.269138 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.269144 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.269150 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:41.269611 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:58:41.269611 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:41.269625 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:58:41.269644 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:41.269824 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:41.269830 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:41.269833 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:41.269892 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 04:58:41.269906 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
I20260808 04:58:41.270241 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.268661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:32909 (local address 127.0.0.1:59604)
0808 04:58:41.269010 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.269018 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.269058 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.269354 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.269355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.269358 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:41.269848 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:58:41.269849 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:41.269856 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:58:41.269871 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:41.270115 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:41.270116 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:41.270116 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 04:58:41.270159 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 04:58:41.270169 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":91,"threads_started":1}
I20260808 04:58:41.270247 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.268864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.3.174.131:40091)
0808 04:58:41.269021 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.269026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.269036 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:41.269059 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.269061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.269070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.269136 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.269710 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.269716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.269716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.269720 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.269754 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.269771 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.269822 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.270151 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.270168 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260808 04:58:41.270442 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.268780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59604 (local address 127.3.174.129:32909)
0808 04:58:41.268942 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.268947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.269142 (+ 195us) server_negotiation.cc:407] Connection header received
0808 04:58:41.269176 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.269179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.269189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.269267 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.269952 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.269961 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.269962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.269966 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.270001 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.270030 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.270088 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.270340 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.270359 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260808 04:58:41.270963 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.268458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:37683 (local address 127.0.0.1:51096)
0808 04:58:41.268872 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.268882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.268925 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.269900 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.269901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.269904 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:41.270317 (+ 413us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:58:41.270317 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:41.270326 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:58:41.270342 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:41.270833 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:41.270834 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:41.270835 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 04:58:41.270889 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 04:58:41.270901 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":2,"spinlock_wait_cycles":896,"thread_start_us":81,"threads_started":1}
I20260808 04:58:41.271018 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.268574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.3.174.130:37683)
0808 04:58:41.268976 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.268980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.268992 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:58:41.269021 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.269023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.269032 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.269115 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.270700 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.270704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.270704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.270708 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.270737 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.270752 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.270828 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.270905 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.270923 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":89,"threads_started":1}
I20260808 04:58:41.270946 12535 tablet_service.cc:1467] Processing CreateTablet for tablet 25278b311da24a8ab81cabd9a09a3046 (DEFAULT_TABLE table=test-table [id=23276d43ebf04a53a582a66f542f94f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:41.271080 12106 tablet_service.cc:1467] Processing CreateTablet for tablet 25278b311da24a8ab81cabd9a09a3046 (DEFAULT_TABLE table=test-table [id=23276d43ebf04a53a582a66f542f94f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:41.271327 12535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25278b311da24a8ab81cabd9a09a3046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:41.271440 12106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25278b311da24a8ab81cabd9a09a3046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:41.271689 12316 tablet_service.cc:1467] Processing CreateTablet for tablet 25278b311da24a8ab81cabd9a09a3046 (DEFAULT_TABLE table=test-table [id=23276d43ebf04a53a582a66f542f94f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:41.272001 12316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25278b311da24a8ab81cabd9a09a3046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:41.274055 12683 tablet_bootstrap.cc:492] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca: Bootstrap starting.
I20260808 04:58:41.274492 12686 tablet_bootstrap.cc:492] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: Bootstrap starting.
I20260808 04:58:41.275020 12687 tablet_bootstrap.cc:492] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: Bootstrap starting.
I20260808 04:58:41.275163 12686 tablet_bootstrap.cc:654] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:41.274981 12683 tablet_bootstrap.cc:654] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:41.275517 12686 log.cc:824] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:41.275671 12687 tablet_bootstrap.cc:654] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:41.275658 12683 log.cc:824] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:41.276008 12687 log.cc:824] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:41.276439 12686 tablet_bootstrap.cc:492] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: No bootstrap required, opened a new log
I20260808 04:58:41.276515 12686 ts_tablet_manager.cc:1397] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:41.276732 12687 tablet_bootstrap.cc:492] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: No bootstrap required, opened a new log
I20260808 04:58:41.276789 12687 ts_tablet_manager.cc:1397] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:41.277700 12683 tablet_bootstrap.cc:492] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca: No bootstrap required, opened a new log
I20260808 04:58:41.277776 12683 ts_tablet_manager.cc:1397] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 04:58:41.278826 12687 raft_consensus.cc:348] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } }
I20260808 04:58:41.278957 12687 raft_consensus.cc:374] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:41.278990 12687 raft_consensus.cc:729] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6393056859f44d0391e45e9faf3d63b1, State: Initialized, Role: FOLLOWER
I20260808 04:58:41.278955 12686 raft_consensus.cc:348] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } }
I20260808 04:58:41.279122 12686 raft_consensus.cc:374] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:41.279124 12687 consensus_queue.cc:260] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } }
I20260808 04:58:41.279181 12686 raft_consensus.cc:729] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80524c6f6297426ba7abf7b68b4a9bdc, State: Initialized, Role: FOLLOWER
I20260808 04:58:41.279325 12686 consensus_queue.cc:260] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } }
I20260808 04:58:41.279392 12687 ts_tablet_manager.cc:1428] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 04:58:41.279397 12419 heartbeater.cc:503] Master 127.3.174.190:46487 was elected leader, sending a full tablet report...
I20260808 04:58:41.279610 12661 heartbeater.cc:503] Master 127.3.174.190:46487 was elected leader, sending a full tablet report...
I20260808 04:58:41.279613 12686 ts_tablet_manager.cc:1428] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 04:58:41.280393 12683 raft_consensus.cc:348] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } }
I20260808 04:58:41.280562 12683 raft_consensus.cc:374] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:41.280609 12683 raft_consensus.cc:729] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abc763c858904e398fae375c119e21ca, State: Initialized, Role: FOLLOWER
I20260808 04:58:41.280730 12683 consensus_queue.cc:260] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } }
I20260808 04:58:41.280993 12683 ts_tablet_manager.cc:1428] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 04:58:41.281013 12212 heartbeater.cc:503] Master 127.3.174.190:46487 was elected leader, sending a full tablet report...
I20260808 04:58:41.343703 12693 raft_consensus.cc:482] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:41.343837 12693 raft_consensus.cc:504] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } }
I20260808 04:58:41.344247 12693 leader_election.cc:288] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80524c6f6297426ba7abf7b68b4a9bdc (127.3.174.131:40091), abc763c858904e398fae375c119e21ca (127.3.174.129:32909)
I20260808 04:58:41.345589 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.344527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:37511 (local address 127.3.174.131:40091)
0808 04:58:41.344572 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.344573 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.344584 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:58:41.344677 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.344678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.344683 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.344733 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.345360 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.345362 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.345362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.345367 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.345388 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.345394 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.345452 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.345518 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.345531 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:58:41.345611 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.344485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:40091 (local address 127.3.174.130:37511)
0808 04:58:41.344540 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.344552 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.344600 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.344754 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.344756 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.344759 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:41.345228 (+ 469us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:58:41.345228 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:41.345237 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:58:41.345253 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:41.345450 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:41.345451 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:41.345451 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 04:58:41.345515 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 04:58:41.345531 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 04:58:41.345790 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.344605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:32909 (local address 127.3.174.130:43729)
0808 04:58:41.344827 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.344835 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.344874 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.344997 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.344997 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.345000 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:41.345419 (+ 419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:58:41.345419 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:41.345426 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:58:41.345437 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:41.345630 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:41.345631 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:41.345631 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 04:58:41.345704 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 04:58:41.345717 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":151,"threads_started":1}
I20260808 04:58:41.345809 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.344699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:43729 (local address 127.3.174.129:32909)
0808 04:58:41.344760 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.344762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.344882 (+ 120us) server_negotiation.cc:407] Connection header received
0808 04:58:41.344928 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.344929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.344935 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.344981 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.345527 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.345530 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.345530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.345536 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.345555 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.345561 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.345617 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.345724 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.345748 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:58:41.346220 12138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25278b311da24a8ab81cabd9a09a3046" candidate_uuid: "6393056859f44d0391e45e9faf3d63b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abc763c858904e398fae375c119e21ca" is_pre_election: true
I20260808 04:58:41.346242 12573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25278b311da24a8ab81cabd9a09a3046" candidate_uuid: "6393056859f44d0391e45e9faf3d63b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" is_pre_election: true
I20260808 04:58:41.346359 12573 raft_consensus.cc:2454] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6393056859f44d0391e45e9faf3d63b1 in term 0.
I20260808 04:58:41.346374 12138 raft_consensus.cc:2454] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6393056859f44d0391e45e9faf3d63b1 in term 0.
I20260808 04:58:41.346592 12242 leader_election.cc:302] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6393056859f44d0391e45e9faf3d63b1, 80524c6f6297426ba7abf7b68b4a9bdc; no voters:
I20260808 04:58:41.346747 12693 raft_consensus.cc:2781] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:41.346832 12693 raft_consensus.cc:482] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:41.346853 12693 raft_consensus.cc:3037] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:41.347581 12693 raft_consensus.cc:504] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } }
I20260808 04:58:41.347700 12693 leader_election.cc:288] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [CANDIDATE]: Term 1 election: Requested vote from peers 80524c6f6297426ba7abf7b68b4a9bdc (127.3.174.131:40091), abc763c858904e398fae375c119e21ca (127.3.174.129:32909)
I20260808 04:58:41.347882 12573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25278b311da24a8ab81cabd9a09a3046" candidate_uuid: "6393056859f44d0391e45e9faf3d63b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80524c6f6297426ba7abf7b68b4a9bdc"
I20260808 04:58:41.347910 12138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25278b311da24a8ab81cabd9a09a3046" candidate_uuid: "6393056859f44d0391e45e9faf3d63b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abc763c858904e398fae375c119e21ca"
I20260808 04:58:41.347980 12573 raft_consensus.cc:3037] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:41.347980 12138 raft_consensus.cc:3037] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:41.348940 12138 raft_consensus.cc:2454] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6393056859f44d0391e45e9faf3d63b1 in term 1.
I20260808 04:58:41.348973 12573 raft_consensus.cc:2454] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6393056859f44d0391e45e9faf3d63b1 in term 1.
I20260808 04:58:41.349108 12242 leader_election.cc:302] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6393056859f44d0391e45e9faf3d63b1, abc763c858904e398fae375c119e21ca; no voters:
I20260808 04:58:41.349222 12693 raft_consensus.cc:2781] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:41.349416 12693 raft_consensus.cc:686] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 1 LEADER]: Becoming Leader. State: Replica: 6393056859f44d0391e45e9faf3d63b1, State: Running, Role: LEADER
I20260808 04:58:41.349520 12693 consensus_queue.cc:237] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } }
I20260808 04:58:41.350356 11869 catalog_manager.cc:5495] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6393056859f44d0391e45e9faf3d63b1 (127.3.174.130). New cstate: current_term: 1 leader_uuid: "6393056859f44d0391e45e9faf3d63b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6393056859f44d0391e45e9faf3d63b1" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 } health_report { overall_health: UNKNOWN } } }
I20260808 04:58:41.365880 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.364944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.3.174.130:37683)
0808 04:58:41.365001 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.365003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.365012 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:41.365068 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.365069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.365082 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.365129 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.365652 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.365653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.365653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.365658 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.365673 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.365677 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.365721 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.365798 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.365810 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 04:58:41.367628 12573 raft_consensus.cc:1261] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [term 1 FOLLOWER]: Refusing update from remote peer 6393056859f44d0391e45e9faf3d63b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:41.367628 12138 raft_consensus.cc:1261] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [term 1 FOLLOWER]: Refusing update from remote peer 6393056859f44d0391e45e9faf3d63b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:41.367945 12693 consensus_queue.cc:1035] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abc763c858904e398fae375c119e21ca" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:41.368074 12697 consensus_queue.cc:1035] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80524c6f6297426ba7abf7b68b4a9bdc" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:41.370594 12702 mvcc.cc:205] Tried to move back new op lower bound from 7316132337118973952 to 7316132337048313856. Current Snapshot: MvccSnapshot[applied={T|T < 7316132337118973952}]
I20260808 04:58:41.373271 11872 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:41.373399 11872 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:41.374114 11872 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6b9c3a5f8dc45ceb8413d79bb9c5b36: Sending DeleteTablet for 3 replicas of tablet 25278b311da24a8ab81cabd9a09a3046
I20260808 04:58:41.374444 12316 tablet_service.cc:1514] Processing DeleteTablet for tablet 25278b311da24a8ab81cabd9a09a3046 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:41 UTC) from {username='slave'} at 127.0.0.1:51096
I20260808 04:58:41.374529 12106 tablet_service.cc:1514] Processing DeleteTablet for tablet 25278b311da24a8ab81cabd9a09a3046 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:41 UTC) from {username='slave'} at 127.0.0.1:59604
I20260808 04:58:41.374775 12718 tablet_replica.cc:330] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca: stopping tablet replica
I20260808 04:58:41.374785 12535 tablet_service.cc:1514] Processing DeleteTablet for tablet 25278b311da24a8ab81cabd9a09a3046 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:41 UTC) from {username='slave'} at 127.0.0.1:48300
I20260808 04:58:41.374881 12718 raft_consensus.cc:2229] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:41.374989 12718 raft_consensus.cc:2258] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:41.375136 12717 tablet_replica.cc:330] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: stopping tablet replica
I20260808 04:58:41.375231 12717 raft_consensus.cc:2229] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:41.375232 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 11989
I20260808 04:58:41.375315 12717 raft_consensus.cc:2258] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:41.375478 12718 ts_tablet_manager.cc:1905] T 25278b311da24a8ab81cabd9a09a3046 P abc763c858904e398fae375c119e21ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:41.375769 12719 tablet_replica.cc:330] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: stopping tablet replica
I20260808 04:58:41.376029 12717 ts_tablet_manager.cc:1905] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:41.376075 12719 raft_consensus.cc:2229] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:41.376200 12719 raft_consensus.cc:2258] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:41.378672 12717 ts_tablet_manager.cc:1918] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:41.378754 12717 log.cc:1196] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/25278b311da24a8ab81cabd9a09a3046
I20260808 04:58:41.378998 12717 ts_tablet_manager.cc:1939] T 25278b311da24a8ab81cabd9a09a3046 P 6393056859f44d0391e45e9faf3d63b1: Deleting consensus metadata
I20260808 04:58:41.379392 11847 catalog_manager.cc:4841] TS 6393056859f44d0391e45e9faf3d63b1 (127.3.174.130:37683): tablet 25278b311da24a8ab81cabd9a09a3046 (table test-table [id=23276d43ebf04a53a582a66f542f94f4]) successfully deleted
I20260808 04:58:41.386252 12719 ts_tablet_manager.cc:1905] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:41.387835 12719 ts_tablet_manager.cc:1918] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:41.387996 12719 log.cc:1196] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/25278b311da24a8ab81cabd9a09a3046
I20260808 04:58:41.388303 12719 ts_tablet_manager.cc:1939] T 25278b311da24a8ab81cabd9a09a3046 P 80524c6f6297426ba7abf7b68b4a9bdc: Deleting consensus metadata
I20260808 04:58:41.388818 11849 catalog_manager.cc:4841] TS 80524c6f6297426ba7abf7b68b4a9bdc (127.3.174.131:40091): tablet 25278b311da24a8ab81cabd9a09a3046 (table test-table [id=23276d43ebf04a53a582a66f542f94f4]) successfully deleted
W20260808 04:58:41.395392 11849 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:32909 (error 108)
I20260808 04:58:41.395577 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 12217
I20260808 04:58:41.400496 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.400303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:32909 (local address 127.0.0.1:59620)
0808 04:58:41.400380 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.400451 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 04:58:41.400631 11849 catalog_manager.cc:4584] TS abc763c858904e398fae375c119e21ca (127.3.174.129:32909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25278b311da24a8ab81cabd9a09a3046: Network error: Client connection negotiation failed: client connection to 127.3.174.129:32909: connect: Connection refused (error 111)
I20260808 04:58:41.423383 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 12425
I20260808 04:58:41.435521 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 11822
2026-08-08T04:58:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (692 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 04:58:41.461166 3770 mini_oidc.cc:152] Starting JWKS server
I20260808 04:58:41.462391 3770 webserver.cc:460] Webserver started at https://127.0.0.1:38581/ using document root <none> and password file <none>
I20260808 04:58:41.462487 3770 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 04:58:41.462654 3770 webserver.cc:460] Webserver started at http://127.0.0.1:44257/ using document root <none> and password file <none>
2026-08-08T04:58:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:41Z Disabled control of system clock
I20260808 04:58:41.471306 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38315
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:42999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:38315
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38581/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/testchainca.cert with env {}
W20260808 04:58:41.553654 12729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:41.553869 12729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:41.553889 12729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:41.555068 12729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:41.555109 12729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:41.555119 12729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:41.555130 12729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:41.555141 12729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 04:58:41.555152 12729 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 04:58:41.555162 12729 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38581/jwks.json
I20260808 04:58:41.557000 12729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:38315
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38581/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:41.557215 12729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:41.557412 12729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:41.562053 12743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:41.562119 12744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:41.562083 12748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:41.562310 12729 server_base.cc:1034] running on GCE node
I20260808 04:58:41.562572 12729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:41.562842 12729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:41.563985 12729 hybrid_clock.cc:648] HybridClock initialized: now 1786165121563968 us; error 34 us; skew 500 ppm
I20260808 04:58:41.565348 12729 webserver.cc:460] Webserver started at http://127.3.174.190:39267/ using document root <none> and password file <none>
I20260808 04:58:41.565584 12729 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:41.565635 12729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:41.565761 12729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:41.566815 12729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "12fdfd4565ed49ddbd916a0132e1bb90"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "9d6e956ef638f0763591e648aa04b519"
server_key_iv: "fb278ff2efe32c648f624bd49bdca0ff"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.567189 12729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "12fdfd4565ed49ddbd916a0132e1bb90"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "9d6e956ef638f0763591e648aa04b519"
server_key_iv: "fb278ff2efe32c648f624bd49bdca0ff"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.568563 12729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:41.569248 12755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.569455 12729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:41.569555 12729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "12fdfd4565ed49ddbd916a0132e1bb90"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "9d6e956ef638f0763591e648aa04b519"
server_key_iv: "fb278ff2efe32c648f624bd49bdca0ff"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.569623 12729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:41.595466 12729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:41.595842 12729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:41.596002 12729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:41.600018 12729 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:38315
I20260808 04:58:41.600061 12846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:38315 every 8 connection(s)
I20260808 04:58:41.600483 12729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:41.601676 12848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:41.604025 12848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90: Bootstrap starting.
I20260808 04:58:41.604642 12848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:41.604918 12848 log.cc:824] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:41.605659 12848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90: No bootstrap required, opened a new log
I20260808 04:58:41.606031 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 12729
I20260808 04:58:41.606127 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:41.606312 3770 external_mini_cluster.cc:1428] Setting key b744bf44dc12da5c1fbbcc62802e9f33
I20260808 04:58:41.607515 12848 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12fdfd4565ed49ddbd916a0132e1bb90" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38315 } }
I20260808 04:58:41.607661 12848 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:41.607692 12848 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12fdfd4565ed49ddbd916a0132e1bb90, State: Initialized, Role: FOLLOWER
I20260808 04:58:41.607851 12848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12fdfd4565ed49ddbd916a0132e1bb90" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38315 } }
I20260808 04:58:41.607933 12848 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:41.607965 12848 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:41.608016 12848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:41.608542 12848 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12fdfd4565ed49ddbd916a0132e1bb90" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38315 } }
I20260808 04:58:41.608637 12848 leader_election.cc:302] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12fdfd4565ed49ddbd916a0132e1bb90; no voters:
I20260808 04:58:41.608798 12848 leader_election.cc:288] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:41.608841 12856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:41.608997 12856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [term 1 LEADER]: Becoming Leader. State: Replica: 12fdfd4565ed49ddbd916a0132e1bb90, State: Running, Role: LEADER
I20260808 04:58:41.609079 12848 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:41.609138 12856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12fdfd4565ed49ddbd916a0132e1bb90" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38315 } }
I20260808 04:58:41.609581 12856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12fdfd4565ed49ddbd916a0132e1bb90. Latest consensus state: current_term: 1 leader_uuid: "12fdfd4565ed49ddbd916a0132e1bb90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12fdfd4565ed49ddbd916a0132e1bb90" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38315 } } }
I20260808 04:58:41.609632 12858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12fdfd4565ed49ddbd916a0132e1bb90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12fdfd4565ed49ddbd916a0132e1bb90" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38315 } } }
I20260808 04:58:41.609760 12858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:41.609965 12856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90 [sys.catalog]: This master's current role is: LEADER
W20260808 04:58:41.611217 12879 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:58:41.611272 12879 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:58:41.611315 12870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:41.611450 12870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:41.612869 12855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.606835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.3.174.190:38315)
0808 04:58:41.607054 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.607062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.607089 (+ 27us) server_negotiation.cc:407] Connection header received
0808 04:58:41.607515 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.607524 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.607593 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.607736 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.607983 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.609476 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.610560 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.611157 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.611291 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:41.612045 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.612050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.612050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.612249 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.612304 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.612320 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.612418 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.612488 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.612652 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260808 04:58:41.613102 12870 catalog_manager.cc:1347] Generated new cluster ID: 3bf58441f18642888fcec7e9c675a1d9
I20260808 04:58:41.613153 12870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:41.613802 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:58:41.624045 12870 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:41.624575 12870 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:41.628504 12870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12fdfd4565ed49ddbd916a0132e1bb90: Generated new TSK 0
I20260808 04:58:41.628670 12870 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 04:58:41.652791 12890 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 04:58:41.652935 12890 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 04:58:41.653115 12855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.637959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.3.174.190:38315)
0808 04:58:41.638016 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.638018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.638161 (+ 143us) server_negotiation.cc:407] Connection header received
0808 04:58:41.638288 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.638289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.638312 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.638367 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0808 04:58:41.638551 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.639030 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.640123 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.640318 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.640365 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:41.640592 (+ 227us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 04:58:41.652884 (+ 12292us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 04:58:41.653050 (+ 166us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44232: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 04:58:41.653162 12855 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44232: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 04:58:41.653753 12889 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:41.637888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38315 (local address 127.0.0.1:44232)
0808 04:58:41.638092 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.638104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.638190 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.638392 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.638393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.638395 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 04:58:41.638479 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:41.638482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.639098 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.639099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:41.640045 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:41.640048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.640385 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.640386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:41.640452 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:41.640477 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 04:58:41.653140 (+ 12663us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 04:58:41.653407 (+ 267us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.174.190:38315: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.3.174.190:38315: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 04:58:41.654244 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 12729
2026-08-08T04:58:41Z chronyd exiting
I20260808 04:58:41.711417 3770 test_util.cc:175] -----------------------------------------------
I20260808 04:58:41.711503 3770 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786165096473501-3770-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (251 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 04:58:41.712392 3770 mini_oidc.cc:152] Starting JWKS server
I20260808 04:58:41.713065 3770 webserver.cc:460] Webserver started at https://127.0.0.1:38537/ using document root <none> and password file <none>
I20260808 04:58:41.713104 3770 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 04:58:41.713254 3770 webserver.cc:460] Webserver started at http://127.0.0.1:42067/ using document root <none> and password file <none>
2026-08-08T04:58:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:41Z Disabled control of system clock
I20260808 04:58:41.722285 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33591
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:33591
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38537/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/testchainca.cert with env {}
W20260808 04:58:41.804785 12901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:41.804993 12901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:41.805024 12901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:41.806234 12901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:41.806279 12901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:41.806296 12901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:41.806313 12901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:41.806327 12901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 04:58:41.806344 12901 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 04:58:41.806366 12901 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38537/jwks.json
I20260808 04:58:41.807904 12901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:33591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38537/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:41.808156 12901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:41.808365 12901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:41.814038 12911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:41.814069 12901 server_base.cc:1034] running on GCE node
W20260808 04:58:41.814282 12908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:41.814435 12907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:41.814728 12901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:41.815024 12901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:41.816192 12901 hybrid_clock.cc:648] HybridClock initialized: now 1786165121816151 us; error 59 us; skew 500 ppm
I20260808 04:58:41.817689 12901 webserver.cc:460] Webserver started at http://127.3.174.190:36421/ using document root <none> and password file <none>
I20260808 04:58:41.817914 12901 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:41.817961 12901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:41.818058 12901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:41.819093 12901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "d3102d0c92634f89bca89cc62fb2d16e"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "d9dc74ee14d26d0ba1340f7a85349812"
server_key_iv: "d08fc624513a20d1e06b6e49c5d068fa"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.819478 12901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "d3102d0c92634f89bca89cc62fb2d16e"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "d9dc74ee14d26d0ba1340f7a85349812"
server_key_iv: "d08fc624513a20d1e06b6e49c5d068fa"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.820652 12901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:41.821276 12918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:41.821476 12901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:41.821559 12901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "d3102d0c92634f89bca89cc62fb2d16e"
format_stamp: "Formatted at 2026-08-08 04:58:41 on dist-test-slave-gjw7"
server_key: "d9dc74ee14d26d0ba1340f7a85349812"
server_key_iv: "d08fc624513a20d1e06b6e49c5d068fa"
server_key_version: "encryptionkey@0"
I20260808 04:58:41.821617 12901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:41.837435 12901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:41.837759 12901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:41.837890 12901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:41.841552 12901 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:33591
I20260808 04:58:41.841594 13013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:33591 every 8 connection(s)
I20260808 04:58:41.841929 12901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:41.842893 13014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:41.845278 13014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e: Bootstrap starting.
I20260808 04:58:41.845907 13014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:41.846143 13014 log.cc:824] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:41.846488 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 12901
I20260808 04:58:41.846589 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:41.846768 3770 external_mini_cluster.cc:1428] Setting key f3f65ec43ef847218b1e2550af1eb238
I20260808 04:58:41.846848 13014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e: No bootstrap required, opened a new log
I20260808 04:58:41.849555 13014 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3102d0c92634f89bca89cc62fb2d16e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33591 } }
I20260808 04:58:41.849717 13014 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:41.849757 13014 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3102d0c92634f89bca89cc62fb2d16e, State: Initialized, Role: FOLLOWER
I20260808 04:58:41.849874 13014 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3102d0c92634f89bca89cc62fb2d16e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33591 } }
I20260808 04:58:41.849948 13014 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:41.849989 13014 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:41.850039 13014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:41.850559 13014 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3102d0c92634f89bca89cc62fb2d16e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33591 } }
I20260808 04:58:41.850661 13014 leader_election.cc:302] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3102d0c92634f89bca89cc62fb2d16e; no voters:
I20260808 04:58:41.850828 13014 leader_election.cc:288] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:41.850852 13020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:41.850958 13020 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [term 1 LEADER]: Becoming Leader. State: Replica: d3102d0c92634f89bca89cc62fb2d16e, State: Running, Role: LEADER
I20260808 04:58:41.851042 13020 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3102d0c92634f89bca89cc62fb2d16e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33591 } }
I20260808 04:58:41.851080 13014 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:41.851419 13021 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3102d0c92634f89bca89cc62fb2d16e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3102d0c92634f89bca89cc62fb2d16e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33591 } } }
I20260808 04:58:41.851501 13021 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:41.851646 13022 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3102d0c92634f89bca89cc62fb2d16e. Latest consensus state: current_term: 1 leader_uuid: "d3102d0c92634f89bca89cc62fb2d16e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3102d0c92634f89bca89cc62fb2d16e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33591 } } }
I20260808 04:58:41.851706 13022 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:41.852301 13027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:41.852977 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.847246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38928 (local address 127.3.174.190:33591)
0808 04:58:41.847496 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.847506 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.847541 (+ 35us) server_negotiation.cc:407] Connection header received
0808 04:58:41.848118 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.848128 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.848208 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.848372 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:41.848576 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.849912 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.850765 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.851336 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.851808 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:41.852210 (+ 402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:41.852214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:41.852214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:41.852440 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:41.852485 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:41.852501 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:41.852602 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:41.852702 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 04:58:41.852801 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":100,"threads_started":1}
I20260808 04:58:41.853217 13027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:41.854312 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:58:41.854753 13027 catalog_manager.cc:1347] Generated new cluster ID: 3ef20c35a0d8449f8934ee4e4eaa9fab
I20260808 04:58:41.854799 13027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:41.867530 13027 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:41.868000 13027 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:41.874197 13027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3102d0c92634f89bca89cc62fb2d16e: Generated new TSK 0
I20260808 04:58:41.874387 13027 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 04:58:41.904935 13055 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 04:58:41.905033 13055 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 04:58:41.905371 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:41.883041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.3.174.190:33591)
0808 04:58:41.883118 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:41.883120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:41.883179 (+ 59us) server_negotiation.cc:407] Connection header received
0808 04:58:41.883320 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:41.883321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:41.883369 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:41.883447 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0808 04:58:41.883664 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.884485 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.885649 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.885970 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.886041 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:41.886215 (+ 174us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 04:58:41.905070 (+ 18855us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 04:58:41.905276 (+ 206us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38944: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 04:58:41.905445 13019 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38944: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 04:58:41.905449 13053 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:41.882910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33591 (local address 127.0.0.1:38944)
0808 04:58:41.883078 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:41.883089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:41.883207 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:41.883501 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:41.883503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:41.883506 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 04:58:41.883564 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:41.883567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.884595 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.884596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:41.885553 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:41.885557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:41.886043 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:41.886044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:41.886119 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:41.886125 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 04:58:41.905208 (+ 19083us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 04:58:41.905347 (+ 139us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.174.190:33591: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260808 04:58:41.905725 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 12901
2026-08-08T04:58:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (252 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 04:58:41.964619 3770 mini_oidc.cc:152] Starting JWKS server
I20260808 04:58:41.965317 3770 webserver.cc:460] Webserver started at https://127.0.0.1:37683/ using document root <none> and password file <none>
I20260808 04:58:41.965374 3770 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 04:58:41.965504 3770 webserver.cc:460] Webserver started at http://127.0.0.1:39227/ using document root <none> and password file <none>
2026-08-08T04:58:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:41Z Disabled control of system clock
I20260808 04:58:41.974121 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:36373
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:36373
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37683/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 04:58:42.061160 13064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:42.061429 13064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:42.061455 13064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:42.062592 13064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:42.062625 13064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:42.062636 13064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:42.062647 13064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:42.062659 13064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 04:58:42.062668 13064 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 04:58:42.062678 13064 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37683/jwks.json
I20260808 04:58:42.063990 13064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:36373
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:36373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37683/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:42.064200 13064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:42.064440 13064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:42.069144 13075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:42.069164 13064 server_base.cc:1034] running on GCE node
W20260808 04:58:42.069128 13072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:42.069132 13077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:42.069646 13064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:42.069912 13064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:42.071053 13064 hybrid_clock.cc:648] HybridClock initialized: now 1786165122071030 us; error 36 us; skew 500 ppm
I20260808 04:58:42.072270 13064 webserver.cc:460] Webserver started at http://127.3.174.190:45703/ using document root <none> and password file <none>
I20260808 04:58:42.072479 13064 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:42.072527 13064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:42.072654 13064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:42.073879 13064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "e5424c0f635e401a90579750cd614c3c"
format_stamp: "Formatted at 2026-08-08 04:58:42 on dist-test-slave-gjw7"
server_key: "e381dd9b692c73d4abfc7cb26da7b9f1"
server_key_iv: "a24cbd8447fda40b52066470f8463547"
server_key_version: "encryptionkey@0"
I20260808 04:58:42.074353 13064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "e5424c0f635e401a90579750cd614c3c"
format_stamp: "Formatted at 2026-08-08 04:58:42 on dist-test-slave-gjw7"
server_key: "e381dd9b692c73d4abfc7cb26da7b9f1"
server_key_iv: "a24cbd8447fda40b52066470f8463547"
server_key_version: "encryptionkey@0"
I20260808 04:58:42.075668 13064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:42.076386 13092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:42.076537 13064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:42.076627 13064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "e5424c0f635e401a90579750cd614c3c"
format_stamp: "Formatted at 2026-08-08 04:58:42 on dist-test-slave-gjw7"
server_key: "e381dd9b692c73d4abfc7cb26da7b9f1"
server_key_iv: "a24cbd8447fda40b52066470f8463547"
server_key_version: "encryptionkey@0"
I20260808 04:58:42.076695 13064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:42.098300 13064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:42.098619 13064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:42.098753 13064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:42.102589 13064 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:36373
I20260808 04:58:42.102639 13180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:36373 every 8 connection(s)
I20260808 04:58:42.102969 13064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:42.103720 13181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:42.106747 13181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c: Bootstrap starting.
I20260808 04:58:42.107345 13181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:42.107630 13181 log.cc:824] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:42.108373 13181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c: No bootstrap required, opened a new log
I20260808 04:58:42.108407 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 13064
I20260808 04:58:42.108510 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:42.108733 3770 external_mini_cluster.cc:1428] Setting key c9abf7b1430659fe81d65698478d93db
I20260808 04:58:42.110402 13181 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5424c0f635e401a90579750cd614c3c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36373 } }
I20260808 04:58:42.110589 13181 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:42.110634 13181 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5424c0f635e401a90579750cd614c3c, State: Initialized, Role: FOLLOWER
I20260808 04:58:42.110785 13181 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5424c0f635e401a90579750cd614c3c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36373 } }
I20260808 04:58:42.110878 13181 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:42.110926 13181 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:42.110980 13181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:42.111536 13181 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5424c0f635e401a90579750cd614c3c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36373 } }
I20260808 04:58:42.111634 13181 leader_election.cc:302] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5424c0f635e401a90579750cd614c3c; no voters:
I20260808 04:58:42.111810 13181 leader_election.cc:288] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:42.111842 13196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:42.111964 13196 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [term 1 LEADER]: Becoming Leader. State: Replica: e5424c0f635e401a90579750cd614c3c, State: Running, Role: LEADER
I20260808 04:58:42.112066 13196 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5424c0f635e401a90579750cd614c3c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36373 } }
I20260808 04:58:42.112094 13181 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:42.112468 13198 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5424c0f635e401a90579750cd614c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5424c0f635e401a90579750cd614c3c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36373 } } }
I20260808 04:58:42.112468 13199 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5424c0f635e401a90579750cd614c3c. Latest consensus state: current_term: 1 leader_uuid: "e5424c0f635e401a90579750cd614c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5424c0f635e401a90579750cd614c3c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36373 } } }
I20260808 04:58:42.112583 13198 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:42.112587 13199 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:42.113451 13203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:42.113978 13203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:42.115548 13203 catalog_manager.cc:1347] Generated new cluster ID: c1096ae1337048d7a1f1c0e040309872
I20260808 04:58:42.115624 13203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:42.115787 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:42.109264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39610 (local address 127.3.174.190:36373)
0808 04:58:42.109472 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:42.109479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:42.109503 (+ 24us) server_negotiation.cc:407] Connection header received
0808 04:58:42.110390 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:42.110400 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:42.110506 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:42.110648 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:42.110898 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.112405 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.113549 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.114150 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.114274 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:42.115018 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:42.115023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:42.115023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:42.115221 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:42.115258 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:42.115271 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:42.115335 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:42.115435 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 04:58:42.115517 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260808 04:58:42.116695 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:58:42.154389 13203 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:42.154935 13203 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:42.159073 13203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5424c0f635e401a90579750cd614c3c: Generated new TSK 0
I20260808 04:58:42.159260 13203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:58:42.173563 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:42.169796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39618 (local address 127.3.174.190:36373)
0808 04:58:42.169857 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:42.169859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:42.169963 (+ 104us) server_negotiation.cc:407] Connection header received
0808 04:58:42.170043 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:42.170044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:42.170056 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:42.170121 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0808 04:58:42.170264 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.170726 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.171912 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.172105 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.172177 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:42.172381 (+ 204us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 04:58:42.173347 (+ 966us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 04:58:42.173481 (+ 134us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39618: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 04:58:42.173624 13194 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39618: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 04:58:42.173928 13232 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:42.169724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:36373 (local address 127.0.0.1:39618)
0808 04:58:42.169891 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:42.169903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:42.169972 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:42.170134 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:42.170135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:42.170137 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 04:58:42.170201 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:42.170203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.170819 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.170820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:42.171814 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:42.171819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.172192 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.172193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:42.172256 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:42.172263 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 04:58:42.173455 (+ 1192us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 04:58:42.173564 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.174.190:36373: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260808 04:58:42.174161 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 13064
E20260808 04:58:42.199477 13233 webserver.cc:533] Webserver: sslize failed:
E20260808 04:58:42.199574 13233 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T04:58:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (264 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T04:58:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:42Z Disabled control of system clock
I20260808 04:58:42.237995 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:42193
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:38173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:42193
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/testchainca.cert with env {}
W20260808 04:58:42.320853 13241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:42.321049 13241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:42.321071 13241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:42.322227 13241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:42.322268 13241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:42.322279 13241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:42.322289 13241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:42.322300 13241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 04:58:42.322311 13241 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 04:58:42.322320 13241 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 04:58:42.323769 13241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:38173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:42193
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:42193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:42.323973 13241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:42.324136 13241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:42.328826 13251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:42.328797 13248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:42.328832 13241 server_base.cc:1034] running on GCE node
W20260808 04:58:42.328787 13249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:42.329337 13241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:42.329631 13241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:42.330783 13241 hybrid_clock.cc:648] HybridClock initialized: now 1786165122330764 us; error 37 us; skew 500 ppm
I20260808 04:58:42.332093 13241 webserver.cc:460] Webserver started at http://127.3.174.190:38625/ using document root <none> and password file <none>
I20260808 04:58:42.332315 13241 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:42.332362 13241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:42.332482 13241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:42.333545 13241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "2016bda915824e428acfe6e95de1a754"
format_stamp: "Formatted at 2026-08-08 04:58:42 on dist-test-slave-gjw7"
server_key: "9ac223e98c13f7568a68ad0bb958bf74"
server_key_iv: "04c3be3e42da8825d78fafccd9f012d3"
server_key_version: "encryptionkey@0"
I20260808 04:58:42.333863 13241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "2016bda915824e428acfe6e95de1a754"
format_stamp: "Formatted at 2026-08-08 04:58:42 on dist-test-slave-gjw7"
server_key: "9ac223e98c13f7568a68ad0bb958bf74"
server_key_iv: "04c3be3e42da8825d78fafccd9f012d3"
server_key_version: "encryptionkey@0"
I20260808 04:58:42.335042 13241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:42.335747 13262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:42.335886 13241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:42.335978 13241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "2016bda915824e428acfe6e95de1a754"
format_stamp: "Formatted at 2026-08-08 04:58:42 on dist-test-slave-gjw7"
server_key: "9ac223e98c13f7568a68ad0bb958bf74"
server_key_iv: "04c3be3e42da8825d78fafccd9f012d3"
server_key_version: "encryptionkey@0"
I20260808 04:58:42.336050 13241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:42.350482 13241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:42.350762 13241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:42.350885 13241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:42.354799 13241 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:42193
I20260808 04:58:42.354847 13344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:42193 every 8 connection(s)
I20260808 04:58:42.355168 13241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:42.356118 13346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:42.358548 13346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754: Bootstrap starting.
I20260808 04:58:42.359128 13346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:42.359416 13346 log.cc:824] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:42.360054 13346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754: No bootstrap required, opened a new log
I20260808 04:58:42.361791 13346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2016bda915824e428acfe6e95de1a754" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42193 } }
I20260808 04:58:42.361935 13346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:42.361959 13346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2016bda915824e428acfe6e95de1a754, State: Initialized, Role: FOLLOWER
I20260808 04:58:42.362107 13346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2016bda915824e428acfe6e95de1a754" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42193 } }
I20260808 04:58:42.362185 13346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:42.362208 13346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:42.362263 13346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:42.362390 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 13241
I20260808 04:58:42.362486 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:42.362648 3770 external_mini_cluster.cc:1428] Setting key b0e809c3a639dd7ca04287219372955e
I20260808 04:58:42.362901 13346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2016bda915824e428acfe6e95de1a754" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42193 } }
I20260808 04:58:42.363016 13346 leader_election.cc:302] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2016bda915824e428acfe6e95de1a754; no voters:
I20260808 04:58:42.363189 13346 leader_election.cc:288] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:42.363363 13359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:42.363454 13346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:42.363500 13359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [term 1 LEADER]: Becoming Leader. State: Replica: 2016bda915824e428acfe6e95de1a754, State: Running, Role: LEADER
I20260808 04:58:42.363597 13359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2016bda915824e428acfe6e95de1a754" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42193 } }
I20260808 04:58:42.364151 13368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2016bda915824e428acfe6e95de1a754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2016bda915824e428acfe6e95de1a754" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42193 } } }
I20260808 04:58:42.364243 13368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:42.364380 13368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2016bda915824e428acfe6e95de1a754. Latest consensus state: current_term: 1 leader_uuid: "2016bda915824e428acfe6e95de1a754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2016bda915824e428acfe6e95de1a754" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42193 } } }
I20260808 04:58:42.364436 13368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:42.364558 13382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:42.365051 13382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:42.368378 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:42.363090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.3.174.190:42193)
0808 04:58:42.363327 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:42.363337 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:42.363371 (+ 34us) server_negotiation.cc:407] Connection header received
0808 04:58:42.363490 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:42.363497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:42.363571 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:42.364018 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:42.364125 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.365562 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.366433 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.366835 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.366921 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:42.367609 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:42.367613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:42.367613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:58:42.367848 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:42.367894 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:58:42.367910 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:42.368021 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:42.368105 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 04:58:42.368187 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":78,"threads_started":1}
I20260808 04:58:42.368438 13382 catalog_manager.cc:1347] Generated new cluster ID: 141cec34dced4e4e8a1f47b3588cf59c
I20260808 04:58:42.368484 13382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:42.369349 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 04:58:42.407227 13382 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:42.407790 13382 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:42.410985 13382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2016bda915824e428acfe6e95de1a754: Generated new TSK 0
I20260808 04:58:42.411161 13382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:58:42.504762 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:42.419729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.3.174.190:42193)
0808 04:58:42.419781 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:42.419783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:42.419869 (+ 86us) server_negotiation.cc:407] Connection header received
0808 04:58:42.419948 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:42.419949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:42.419974 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:42.420029 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0808 04:58:42.420163 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.420640 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.421475 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.421664 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.421739 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:42.421888 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 04:58:42.504491 (+ 82603us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 04:58:42.504686 (+ 195us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52574: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 04:58:42.504848 13358 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52574: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 04:58:42.504885 13405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:42.419671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:42193 (local address 127.0.0.1:52574)
0808 04:58:42.419808 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:42.419819 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:42.419879 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:42.420037 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:42.420038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:42.420041 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 04:58:42.420103 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:42.420106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.420748 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.420749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:42.421404 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:42.421406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:42.421737 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:42.421739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:42.421795 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:42.421801 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 04:58:42.504645 (+ 82844us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 04:58:42.504790 (+ 145us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.174.190:42193: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260808 04:58:42.505312 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 13241
2026-08-08T04:58:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (300 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:42 dist-test-slave-gjw7 krb5kdc[13421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:42 dist-test-slave-gjw7 krb5kdc[13421](info): set up 2 sockets
Aug 08 04:58:42 dist-test-slave-gjw7 krb5kdc[13421](info): commencing operation
krb5kdc: starting...
W20260808 04:58:43.898087 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:43 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:44.015349 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:43993
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:43993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:44.096959 13438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:44.097167 13438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:44.097198 13438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:44.098347 13438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:44.098382 13438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:44.098394 13438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:44.098405 13438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:44.098416 13438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:44.099799 13438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:43993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:43993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:44.100001 13438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:44.100171 13438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:44.104831 13447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:44.104838 13454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:44.104890 13451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:44.104892 13438 server_base.cc:1034] running on GCE node
I20260808 04:58:44.105342 13438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:44.105615 13438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:44.106756 13438 hybrid_clock.cc:648] HybridClock initialized: now 1786165124106743 us; error 26 us; skew 500 ppm
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.109449 13438 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:44.110000 13438 webserver.cc:460] Webserver started at http://127.3.174.190:43377/ using document root <none> and password file <none>
I20260808 04:58:44.110234 13438 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:44.110288 13438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:44.110438 13438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:44.111557 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "00a9ad49dfbd4408907d7c55f3a11cfd"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "d1f9b7915b00242fa4a213353698d033"
server_key_iv: "351bab72bb3cbd6086306421b7889284"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.112006 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "00a9ad49dfbd4408907d7c55f3a11cfd"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "d1f9b7915b00242fa4a213353698d033"
server_key_iv: "351bab72bb3cbd6086306421b7889284"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.113313 13438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:44.114037 13470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.114187 13438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:44.114274 13438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "00a9ad49dfbd4408907d7c55f3a11cfd"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "d1f9b7915b00242fa4a213353698d033"
server_key_iv: "351bab72bb3cbd6086306421b7889284"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.114344 13438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:44.134701 13438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:44.136582 13438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:44.136727 13438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:44.140738 13438 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:43993
I20260808 04:58:44.140793 13561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:43993 every 8 connection(s)
I20260808 04:58:44.141136 13438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:44.142066 13562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:44.144383 13562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd: Bootstrap starting.
I20260808 04:58:44.145013 13562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:44.145282 13562 log.cc:824] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:44.145983 13562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd: No bootstrap required, opened a new log
I20260808 04:58:44.147576 13562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9ad49dfbd4408907d7c55f3a11cfd" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43993 } }
I20260808 04:58:44.147718 13562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:44.147744 13562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a9ad49dfbd4408907d7c55f3a11cfd, State: Initialized, Role: FOLLOWER
I20260808 04:58:44.147856 13562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9ad49dfbd4408907d7c55f3a11cfd" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43993 } }
I20260808 04:58:44.147908 13562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:44.147926 13562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:44.147957 13562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:44.148432 13562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9ad49dfbd4408907d7c55f3a11cfd" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43993 } }
I20260808 04:58:44.148520 13562 leader_election.cc:302] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00a9ad49dfbd4408907d7c55f3a11cfd; no voters:
I20260808 04:58:44.148653 13562 leader_election.cc:288] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:44.148712 13570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:44.148862 13570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [term 1 LEADER]: Becoming Leader. State: Replica: 00a9ad49dfbd4408907d7c55f3a11cfd, State: Running, Role: LEADER
I20260808 04:58:44.148877 13562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:44.148954 13570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9ad49dfbd4408907d7c55f3a11cfd" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43993 } }
I20260808 04:58:44.149360 13572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00a9ad49dfbd4408907d7c55f3a11cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9ad49dfbd4408907d7c55f3a11cfd" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43993 } } }
I20260808 04:58:44.149446 13572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:44.149657 13573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00a9ad49dfbd4408907d7c55f3a11cfd. Latest consensus state: current_term: 1 leader_uuid: "00a9ad49dfbd4408907d7c55f3a11cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9ad49dfbd4408907d7c55f3a11cfd" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43993 } } }
I20260808 04:58:44.149717 13573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:44.149829 13587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:44.150403 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 13438
I20260808 04:58:44.150490 13587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:44.150514 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:44.150705 3770 external_mini_cluster.cc:1428] Setting key fbd39dbb712a0e058e88391f1cb2fa19
I20260808 04:58:44.152060 13587 catalog_manager.cc:1347] Generated new cluster ID: 50b7e368ef1a46e4acac14c2622f9de0
I20260808 04:58:44.152110 13587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:44.159423 13587 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:44.160127 13587 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:44.170926 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.151176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52940 (local address 127.3.174.190:43993)
0808 04:58:44.151486 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.151496 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.151525 (+ 29us) server_negotiation.cc:407] Connection header received
0808 04:58:44.151639 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.151645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.151713 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.151849 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.152578 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.153591 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.154315 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.154760 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.154871 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.157052 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:44.157057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:44.157057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:44.157071 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:44.158594 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.158918 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.158922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.158922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.159033 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.169343 (+ 10310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.169345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.169345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.169780 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:44.170385 (+ 605us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:44.170469 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 04:58:44.170640 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":83,"threads_started":1}
I20260808 04:58:44.172786 13587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00a9ad49dfbd4408907d7c55f3a11cfd: Generated new TSK 0
I20260808 04:58:44.172966 13587 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:44.201930 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:43993
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:44.285449 13611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:44.285645 13611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:44.285667 13611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:44.286835 13611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:44.286873 13611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:44.286900 13611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:44.288391 13611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:43993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:44.288628 13611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:44.288867 13611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:44.293769 13625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:44.293783 13622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:44.293778 13621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:44.294466 13611 server_base.cc:1034] running on GCE node
I20260808 04:58:44.294644 13611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:44.294839 13611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:44.295989 13611 hybrid_clock.cc:648] HybridClock initialized: now 1786165124295976 us; error 27 us; skew 500 ppm
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.298601 13611 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:44.299052 13611 webserver.cc:460] Webserver started at http://127.3.174.129:42609/ using document root <none> and password file <none>
I20260808 04:58:44.299283 13611 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:44.299332 13611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:44.299458 13611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:44.300529 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "60056d2556864bf6ad378f886774a5ae"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "eec8a6fcb9bdd22cb5753d8184f2a0c4"
server_key_iv: "36d65b4c9f6466c1c9294479cade8201"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.300859 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "60056d2556864bf6ad378f886774a5ae"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "eec8a6fcb9bdd22cb5753d8184f2a0c4"
server_key_iv: "36d65b4c9f6466c1c9294479cade8201"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.302271 13611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:44.302928 13636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.303073 13611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:44.303157 13611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "60056d2556864bf6ad378f886774a5ae"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "eec8a6fcb9bdd22cb5753d8184f2a0c4"
server_key_iv: "36d65b4c9f6466c1c9294479cade8201"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.303260 13611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:44.311728 13611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:44.313448 13611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:44.313575 13611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:44.313792 13611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:44.314081 13611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:44.314113 13611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.314141 13611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:44.314164 13611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.319796 13611 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:37647
I20260808 04:58:44.319859 13815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:37647 every 8 connection(s)
I20260808 04:58:44.320147 13611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:44.327046 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 13611
I20260808 04:58:44.327157 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:44.327354 3770 external_mini_cluster.cc:1428] Setting key c4e28cd69397f8069f5f17abaed88aee
I20260808 04:58:44.329126 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.320825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:43181 (local address 127.3.174.190:43993)
0808 04:58:44.320907 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.320909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.321057 (+ 148us) server_negotiation.cc:407] Connection header received
0808 04:58:44.322038 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.322039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.322049 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.322100 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.322664 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.323223 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.324195 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.324427 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.324478 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.326492 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:44.326493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:44.326494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:44.326500 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:44.327819 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.328233 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.328234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.328235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.328292 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.328512 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.328513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.328513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.328624 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:44.328682 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:44.328947 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 04:58:44.329036 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 04:58:44.329430 13819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.320800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:43993 (local address 127.3.174.129:43181)
0808 04:58:44.320947 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:44.320983 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:44.321943 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:44.322156 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:44.322159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:44.322403 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:44.322585 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:44.322589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.323328 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.323329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:44.324118 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:44.324120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.324493 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.324494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:44.324598 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.325016 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:44.325024 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:44.326397 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:44.327888 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:44.327897 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:44.327899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:44.328157 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:44.328346 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:44.328347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:44.328347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:44.328455 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:44.328679 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:44.328687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:44.328826 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 04:58:44.328914 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 04:58:44.329067 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260808 04:58:44.329984 13816 heartbeater.cc:347] Connected to a master server at 127.3.174.190:43993
I20260808 04:58:44.330090 13816 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:44.330287 13816 heartbeater.cc:511] Master 127.3.174.190:43993 requested a full tablet report, sending...
I20260808 04:58:44.330852 13504 ts_manager.cc:194] Registered new tserver with Master: 60056d2556864bf6ad378f886774a5ae (127.3.174.129:37647)
I20260808 04:58:44.331457 13504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:43181
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:44.354228 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:43993
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:44.435128 13822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:44.435330 13822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:44.435353 13822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:44.436471 13822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:44.436507 13822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:44.436532 13822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:44.437947 13822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:43993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:44.438127 13822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:44.438336 13822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:44.443140 13828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:44.443146 13834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:44.443157 13829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:44.443440 13822 server_base.cc:1034] running on GCE node
I20260808 04:58:44.443651 13822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:44.443861 13822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:44.445012 13822 hybrid_clock.cc:648] HybridClock initialized: now 1786165124444997 us; error 23 us; skew 500 ppm
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.447561 13822 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:44.447978 13822 webserver.cc:460] Webserver started at http://127.3.174.130:44087/ using document root <none> and password file <none>
I20260808 04:58:44.448151 13822 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:44.448199 13822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:44.448302 13822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:44.449424 13822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "bf75b2d3371d44cc9630347c59c60742"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "585fcaf16a2672ed9ef31cefc19f23eb"
server_key_iv: "7c743b86b28f66e7979f20871de74fee"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.449780 13822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "bf75b2d3371d44cc9630347c59c60742"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "585fcaf16a2672ed9ef31cefc19f23eb"
server_key_iv: "7c743b86b28f66e7979f20871de74fee"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.451092 13822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:44.451877 13846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.452042 13822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:44.452142 13822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "bf75b2d3371d44cc9630347c59c60742"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "585fcaf16a2672ed9ef31cefc19f23eb"
server_key_iv: "7c743b86b28f66e7979f20871de74fee"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.452265 13822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:44.465497 13822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:44.467816 13822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:44.467947 13822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:44.468187 13822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:44.468506 13822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:44.468545 13822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.468570 13822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:44.468597 13822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.474740 13822 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:44193
I20260808 04:58:44.474783 14041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:44193 every 8 connection(s)
I20260808 04:58:44.475085 13822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:58:44.478744 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 13822
I20260808 04:58:44.478833 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:44.478994 3770 external_mini_cluster.cc:1428] Setting key 7275e0db400c58c7b4d936c5ebb509c1
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:44.484578 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.475812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:38639 (local address 127.3.174.190:43993)
0808 04:58:44.475856 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.475859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.476060 (+ 201us) server_negotiation.cc:407] Connection header received
0808 04:58:44.476975 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.476976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.476986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.477036 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.477645 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.478145 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.479246 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.479494 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.479546 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.481813 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:44.481814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:44.481814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:44.481822 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:44.483188 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.483581 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.483583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.483583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.483652 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.483863 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.483863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.483864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.484015 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:44.484090 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:44.484405 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 04:58:44.484487 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 04:58:44.484659 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.475776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:43993 (local address 127.3.174.130:38639)
0808 04:58:44.475942 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:44.475979 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:44.476888 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:44.477096 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:44.477100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:44.477385 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:44.477574 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:44.477580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.478264 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.478265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:44.479148 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:44.479151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.479579 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.479580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:44.479729 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.480220 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:44.480230 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:44.481724 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:44.483282 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:44.483289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:44.483291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:44.483507 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:44.483712 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:44.483712 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:44.483712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:44.483809 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:44.484079 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:44.484082 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:44.484252 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 04:58:44.484362 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 04:58:44.484493 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260808 04:58:44.485152 14043 heartbeater.cc:347] Connected to a master server at 127.3.174.190:43993
I20260808 04:58:44.485249 14043 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:44.485445 14043 heartbeater.cc:511] Master 127.3.174.190:43993 requested a full tablet report, sending...
I20260808 04:58:44.485844 13504 ts_manager.cc:194] Registered new tserver with Master: bf75b2d3371d44cc9630347c59c60742 (127.3.174.130:44193)
I20260808 04:58:44.486343 13504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:38639
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:44.503974 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:43993
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:44.584847 14051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:44.585034 14051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:44.585064 14051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:44.586229 14051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:44.586269 14051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:44.586302 14051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:44.587719 14051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:43993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:44.587957 14051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:44.588160 14051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:44.592769 14071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:44.592769 14066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:44.592784 14068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:44.593118 14051 server_base.cc:1034] running on GCE node
I20260808 04:58:44.593325 14051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:44.593520 14051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:44.594655 14051 hybrid_clock.cc:648] HybridClock initialized: now 1786165124594645 us; error 26 us; skew 500 ppm
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.597059 14051 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:44.597491 14051 webserver.cc:460] Webserver started at http://127.3.174.131:34427/ using document root <none> and password file <none>
I20260808 04:58:44.597713 14051 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:44.597779 14051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:44.597918 14051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:44.598877 14051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "02f300342f7c497f9a76bac4511ad928"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "bdc5d2348b80d05dff544fed4e1765b0"
server_key_iv: "2a0f98a875be0cfac5e906dc69d3fbbf"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.599200 14051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "02f300342f7c497f9a76bac4511ad928"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "bdc5d2348b80d05dff544fed4e1765b0"
server_key_iv: "2a0f98a875be0cfac5e906dc69d3fbbf"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.600384 14051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:44.601029 14084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.601181 14051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:44.601262 14051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "02f300342f7c497f9a76bac4511ad928"
format_stamp: "Formatted at 2026-08-08 04:58:44 on dist-test-slave-gjw7"
server_key: "bdc5d2348b80d05dff544fed4e1765b0"
server_key_iv: "2a0f98a875be0cfac5e906dc69d3fbbf"
server_key_version: "encryptionkey@0"
I20260808 04:58:44.601352 14051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:44.615226 14051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:44.617270 14051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:44.617424 14051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:44.617628 14051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:44.617918 14051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:44.617951 14051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.617980 14051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:44.617996 14051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:44.623262 14051 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:32873
I20260808 04:58:44.623330 14263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:32873 every 8 connection(s)
I20260808 04:58:44.623590 14051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 04:58:44.628262 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 14051
I20260808 04:58:44.628341 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:44.628491 3770 external_mini_cluster.cc:1428] Setting key 97eff81ea1aafa77d57e65c7643d4f9a
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:44.632112 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.624228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:43939 (local address 127.3.174.190:43993)
0808 04:58:44.624288 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.624290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.624461 (+ 171us) server_negotiation.cc:407] Connection header received
0808 04:58:44.625356 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.625358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.625369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.625416 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.625945 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.626454 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.627344 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.627517 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.627560 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.629799 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:44.629801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:44.629801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:44.629810 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:44.630876 (+ 1066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.631223 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.631225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.631225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.631306 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.631518 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.631519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.631519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.631629 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:44.631683 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:44.631937 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 04:58:44.632023 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 04:58:44.632243 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.624187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:43993 (local address 127.3.174.131:43939)
0808 04:58:44.624360 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:44.624396 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:44.625247 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:44.625471 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:44.625474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:44.625700 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:44.625885 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:44.625889 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.626513 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.626514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:44.627285 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:44.627287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.627568 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.627569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:44.627664 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.628101 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:44.628110 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:44.629701 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:44.630947 (+ 1246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:44.630953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:44.630955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:44.631157 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:44.631362 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:44.631363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:44.631363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:44.631462 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:44.631682 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:44.631688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:44.631820 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 04:58:44.631905 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 04:58:44.632046 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":109,"threads_started":1}
I20260808 04:58:44.632652 14265 heartbeater.cc:347] Connected to a master server at 127.3.174.190:43993
I20260808 04:58:44.632745 14265 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:44.632882 14265 heartbeater.cc:511] Master 127.3.174.190:43993 requested a full tablet report, sending...
I20260808 04:58:44.633217 13504 ts_manager.cc:194] Registered new tserver with Master: 02f300342f7c497f9a76bac4511ad928 (127.3.174.131:32873)
I20260808 04:58:44.633598 13504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:43939
I20260808 04:58:44.640142 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/test-admin.keytab.
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.655613 3770 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 04:58:44.657327 14278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:44.656467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.3.174.129:37647)
0808 04:58:44.656619 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.656624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.656632 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:44.656657 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.656659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.656672 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.656748 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.657223 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55772: BlockingRecv error: recv got EOF from 127.0.0.1:55772 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
W20260808 04:58:44.657413 3770 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5cc)
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.658329 3770 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.659082 3770 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:44.666270 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.659608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.3.174.129:37647)
0808 04:58:44.659655 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.659657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.659667 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:44.659750 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.659751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.659760 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.659804 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.660187 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.660933 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.661670 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.661982 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.662043 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.663886 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:44.663895 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:44.663896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:44.663902 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:44.665135 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.665513 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.665516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.665516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.665569 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.665740 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.665741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.665741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.665875 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:44.666000 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:44.666096 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 04:58:44.666168 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 04:58:44.667857 14278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:44.667429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.3.174.129:37647)
0808 04:58:44.667480 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.667482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.667490 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:44.667544 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.667545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.667554 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.667595 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.667818 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55792: BlockingRecv error: recv got EOF from 127.0.0.1:55792 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 04:58:44.667954 3770 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5cc' of type FILE
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.668805 3770 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.669548 3770 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:44.675751 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.670001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.3.174.129:37647)
0808 04:58:44.670051 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.670053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.670061 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:44.670107 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.670108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.670116 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.670157 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.670487 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.670968 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.671573 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.671751 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.671792 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.673849 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:44.673850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:44.673850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:44.673857 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:44.674987 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.675258 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.675259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.675259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.675302 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.675423 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.675423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.675423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.675499 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:44.675561 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:44.675629 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 04:58:44.675680 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260808 04:58:44.677017 14278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:44.676593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55816 (local address 127.3.174.129:37647)
0808 04:58:44.676648 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.676649 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.676671 (+ 22us) server_negotiation.cc:407] Connection header received
0808 04:58:44.676718 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.676719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.676725 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.676766 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.676973 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55816: BlockingRecv error: recv got EOF from 127.0.0.1:55816 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 04:58:44.677109 3770 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786165096473501-3770-0/krb5kdc/krb5cc' of type FILE
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.677973 3770 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:44.678725 3770 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[13421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 04:58:44.684522 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:44.679149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55826 (local address 127.3.174.129:37647)
0808 04:58:44.679196 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:44.679198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:44.679255 (+ 57us) server_negotiation.cc:407] Connection header received
0808 04:58:44.679286 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:44.679287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:44.679293 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:44.679336 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:44.679632 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.680078 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.680733 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:44.680914 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:44.680974 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:44.682732 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:44.682733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:44.682734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:44.682741 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:44.683749 (+ 1008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.684011 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.684011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.684011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.684051 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:44.684168 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:44.684168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:44.684169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:44.684260 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:44.684319 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:44.684400 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 04:58:44.684449 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 04:58:44.685094 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 13611
I20260808 04:58:44.699158 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 13822
I20260808 04:58:44.724211 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 14051
I20260808 04:58:44.733361 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 13438
2026-08-08T04:58:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2040 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[14295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[14295](info): set up 2 sockets
Aug 08 04:58:44 dist-test-slave-gjw7 krb5kdc[14295](info): commencing operation
krb5kdc: starting...
W20260808 04:58:45.915922 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:45 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:46Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.190.keytab.
Entry for principal oryx/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.190.keytab.
I20260808 04:58:46.033982 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35583
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.190.keytab
--principal=oryx/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:35583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:46.116650 14315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:46.116858 14315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:46.116880 14315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:46.118011 14315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:46.118047 14315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:46.118059 14315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:46.118070 14315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:46.118081 14315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:46.119491 14315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:35583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.190.keytab
--principal=oryx/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:46.119688 14315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:46.119869 14315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:46.124589 14320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:46.124759 14315 server_base.cc:1034] running on GCE node
W20260808 04:58:46.124606 14325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:46.124578 14321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:46.125116 14315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:46.125443 14315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:46.126585 14315 hybrid_clock.cc:648] HybridClock initialized: now 1786165126126572 us; error 31 us; skew 500 ppm
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:46.128999 14315 init.cc:377] Logged in from keytab as oryx/127.3.174.190@KRBTEST.COM (short username oryx)
I20260808 04:58:46.129505 14315 webserver.cc:460] Webserver started at http://127.3.174.190:43065/ using document root <none> and password file <none>
I20260808 04:58:46.129736 14315 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:46.129787 14315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:46.129920 14315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:46.130978 14315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "8917e6eeb4234234b3b255985ff749e9"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "04b44adc3783318c8e98b286779e1c08"
server_key_iv: "61632b1c6150e6602b7744c3334bb822"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.131333 14315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "8917e6eeb4234234b3b255985ff749e9"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "04b44adc3783318c8e98b286779e1c08"
server_key_iv: "61632b1c6150e6602b7744c3334bb822"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.132576 14315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:46.133230 14338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.133431 14315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:46.133517 14315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "8917e6eeb4234234b3b255985ff749e9"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "04b44adc3783318c8e98b286779e1c08"
server_key_iv: "61632b1c6150e6602b7744c3334bb822"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.133581 14315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:46.147248 14315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:46.149121 14315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:46.149253 14315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:46.152962 14315 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:35583
I20260808 04:58:46.153015 14431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:35583 every 8 connection(s)
I20260808 04:58:46.153409 14315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:46.154409 14434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:46.156702 14434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: Bootstrap starting.
I20260808 04:58:46.157277 14434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:46.157577 14434 log.cc:824] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:46.158327 14434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: No bootstrap required, opened a new log
I20260808 04:58:46.158847 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 14315
I20260808 04:58:46.158964 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:46.159170 3770 external_mini_cluster.cc:1428] Setting key 2e9e60f61da91ba6a4b298ac5db43622
I20260808 04:58:46.160857 14434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } }
I20260808 04:58:46.161015 14434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:46.161052 14434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8917e6eeb4234234b3b255985ff749e9, State: Initialized, Role: FOLLOWER
I20260808 04:58:46.161185 14434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } }
I20260808 04:58:46.161269 14434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:46.161340 14434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:46.161393 14434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:46.162024 14434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } }
I20260808 04:58:46.162130 14434 leader_election.cc:302] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8917e6eeb4234234b3b255985ff749e9; no voters:
I20260808 04:58:46.162278 14434 leader_election.cc:288] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:46.162319 14440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:46.162480 14440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 1 LEADER]: Becoming Leader. State: Replica: 8917e6eeb4234234b3b255985ff749e9, State: Running, Role: LEADER
I20260808 04:58:46.162549 14434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:46.162662 14440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } }
I20260808 04:58:46.163034 14440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8917e6eeb4234234b3b255985ff749e9. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } }
I20260808 04:58:46.163089 14445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } }
I20260808 04:58:46.163203 14440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:46.163213 14445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:46.163497 14457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:46.164311 14457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:46.167896 14457 catalog_manager.cc:1347] Generated new cluster ID: 10dd19f8c9d84032a41440f7466a608b
I20260808 04:58:46.167939 14457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:46.170809 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.159706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56346 (local address 127.3.174.190:35583)
0808 04:58:46.159861 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.159870 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.159900 (+ 30us) server_negotiation.cc:407] Connection header received
0808 04:58:46.160803 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.160811 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.160863 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.160977 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.161843 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.163009 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.164063 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.164760 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.164859 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.167100 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.167104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.167105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.167117 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.169131 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.169538 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.169541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.169542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.169671 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.169861 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.169862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.169863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.170190 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.170374 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.170459 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.170607 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260808 04:58:46.193928 14457 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:46.194670 14457 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:46.199797 14457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: Generated new TSK 0
I20260808 04:58:46.200039 14457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab.
Entry for principal oryx/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab.
I20260808 04:58:46.236860 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35583
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab
--principal=oryx/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:46.320410 14480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:46.320614 14480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:46.320644 14480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:46.321789 14480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:46.321830 14480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:46.321871 14480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:46.323371 14480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab
--principal=oryx/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:46.323616 14480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:46.323817 14480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:46.328629 14486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:46.328657 14492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:46.328679 14495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:46.328717 14480 server_base.cc:1034] running on GCE node
I20260808 04:58:46.329008 14480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:46.329241 14480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:46.330382 14480 hybrid_clock.cc:648] HybridClock initialized: now 1786165126330374 us; error 34 us; skew 500 ppm
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:46.332808 14480 init.cc:377] Logged in from keytab as oryx/127.3.174.129@KRBTEST.COM (short username oryx)
I20260808 04:58:46.333190 14480 webserver.cc:460] Webserver started at http://127.3.174.129:38211/ using document root <none> and password file <none>
I20260808 04:58:46.333431 14480 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:46.333477 14480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:46.333571 14480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:46.334542 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "4f0b750dd22c45b79e0b58838242cf2c"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "05fc0ac777aee52d49f4425482d1f5db"
server_key_iv: "cf3a249b8004bae7285a49f239137bad"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.334836 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "4f0b750dd22c45b79e0b58838242cf2c"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "05fc0ac777aee52d49f4425482d1f5db"
server_key_iv: "cf3a249b8004bae7285a49f239137bad"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.336095 14480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 04:58:46.336802 14507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.336977 14480 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:46.337064 14480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "4f0b750dd22c45b79e0b58838242cf2c"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "05fc0ac777aee52d49f4425482d1f5db"
server_key_iv: "cf3a249b8004bae7285a49f239137bad"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.337122 14480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:46.349924 14480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:46.351670 14480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:46.351785 14480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:46.351966 14480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:46.352244 14480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:46.352275 14480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.352296 14480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:46.352312 14480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.357937 14480 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:35607
I20260808 04:58:46.358007 14716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:35607 every 8 connection(s)
I20260808 04:58:46.358297 14480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:58:46.362718 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 14480
I20260808 04:58:46.362833 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:46.363066 3770 external_mini_cluster.cc:1428] Setting key 2fd620ed5d84cf0763de687ea8fbdff1
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:46.367851 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.358993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:37199 (local address 127.3.174.190:35583)
0808 04:58:46.359049 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.359052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.359227 (+ 175us) server_negotiation.cc:407] Connection header received
0808 04:58:46.359754 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.359756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.359770 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.359820 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.360776 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.361316 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.362508 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.362716 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.362789 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.365257 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.365258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.365258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.365266 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.366555 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.366934 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.366935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.366936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.366997 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.367236 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.367237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.367237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.367369 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.367435 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.367664 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.367758 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 04:58:46.368005 14725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.358960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.129:37199)
0808 04:58:46.359109 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:46.359151 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:46.359652 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:46.359917 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:46.359921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:46.360485 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:46.360673 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.360678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.361409 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.361410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.362376 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.362380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.362813 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.362814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.362961 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.363559 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:46.363569 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:46.365169 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:46.366635 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.366643 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.366645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.366864 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.367062 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.367063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.367063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.367173 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.367429 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:46.367433 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:46.367548 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 04:58:46.367629 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 04:58:46.367753 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260808 04:58:46.368507 14719 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35583
I20260808 04:58:46.368602 14719 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:46.368801 14719 heartbeater.cc:511] Master 127.3.174.190:35583 requested a full tablet report, sending...
I20260808 04:58:46.369423 14365 ts_manager.cc:194] Registered new tserver with Master: 4f0b750dd22c45b79e0b58838242cf2c (127.3.174.129:35607)
I20260808 04:58:46.370353 14365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:37199
WARNING: no policy specified for oryx/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab.
Entry for principal oryx/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab.
I20260808 04:58:46.391664 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35583
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab
--principal=oryx/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:46.476505 14734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:46.476715 14734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:46.476747 14734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:46.477990 14734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:46.478037 14734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:46.478072 14734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:46.479516 14734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab
--principal=oryx/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:46.479748 14734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:46.479965 14734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:46.484838 14739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:46.484834 14742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:46.484859 14740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:46.485107 14734 server_base.cc:1034] running on GCE node
I20260808 04:58:46.485355 14734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:46.485591 14734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:46.486729 14734 hybrid_clock.cc:648] HybridClock initialized: now 1786165126486713 us; error 32 us; skew 500 ppm
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:46.489217 14734 init.cc:377] Logged in from keytab as oryx/127.3.174.130@KRBTEST.COM (short username oryx)
I20260808 04:58:46.489641 14734 webserver.cc:460] Webserver started at http://127.3.174.130:35347/ using document root <none> and password file <none>
I20260808 04:58:46.489825 14734 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:46.489866 14734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:46.489957 14734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:46.490954 14734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "b23c85f2fa75450483b891952ae06569"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "894ab01c556273932f28683a85d5446e"
server_key_iv: "f81f83c93bae1584379f3b125de267f7"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.491279 14734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "b23c85f2fa75450483b891952ae06569"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "894ab01c556273932f28683a85d5446e"
server_key_iv: "f81f83c93bae1584379f3b125de267f7"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.492556 14734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:46.493170 14756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.493345 14734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:46.493434 14734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "b23c85f2fa75450483b891952ae06569"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "894ab01c556273932f28683a85d5446e"
server_key_iv: "f81f83c93bae1584379f3b125de267f7"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.493498 14734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:46.512657 14734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:46.514665 14734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:46.514808 14734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:46.515038 14734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:46.515357 14734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:46.515391 14734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.515421 14734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:46.515445 14734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.521126 14734 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:36689
I20260808 04:58:46.521229 14979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:36689 every 8 connection(s)
I20260808 04:58:46.521538 14734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:46.527488 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 14734
I20260808 04:58:46.527591 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:46.527807 3770 external_mini_cluster.cc:1428] Setting key a3609a367f4859b905024210afff6e44
I20260808 04:58:46.530731 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.522214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:55271 (local address 127.3.174.190:35583)
0808 04:58:46.522295 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.522297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.522474 (+ 177us) server_negotiation.cc:407] Connection header received
0808 04:58:46.522990 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.522991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.523004 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.523056 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.523987 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.524524 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.525546 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.525751 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.525807 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.527970 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.527973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.527973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.527981 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.529403 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.529833 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.529834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.529834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.529880 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.530108 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.530109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.530109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.530200 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.530307 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.530569 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.530644 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 04:58:46.530918 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.522185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.130:55271)
0808 04:58:46.522365 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:46.522401 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:46.522894 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:46.523130 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:46.523135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:46.523695 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:46.523894 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.523899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.524612 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.524613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.525471 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.525474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.525819 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.525820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.525941 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.526392 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:46.526399 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:46.527881 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:46.529472 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.529480 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.529483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.529767 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.529930 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.529931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.529932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.530058 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.530302 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:46.530309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:46.530447 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 04:58:46.530537 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:58:46.530690 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260808 04:58:46.531469 14981 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35583
I20260808 04:58:46.531559 14981 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:46.531737 14981 heartbeater.cc:511] Master 127.3.174.190:35583 requested a full tablet report, sending...
I20260808 04:58:46.532136 14365 ts_manager.cc:194] Registered new tserver with Master: b23c85f2fa75450483b891952ae06569 (127.3.174.130:36689)
I20260808 04:58:46.532492 14365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:55271
WARNING: no policy specified for oryx/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab.
Entry for principal oryx/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab.
I20260808 04:58:46.553683 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35583
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab
--principal=oryx/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:46.635962 14991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:46.636168 14991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:46.636205 14991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:46.637401 14991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:46.637444 14991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:46.637478 14991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:46.638917 14991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab
--principal=oryx/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:46.639146 14991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:46.639360 14991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:46.644176 15007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:46.644263 15010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:46.644181 15008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:46.644470 14991 server_base.cc:1034] running on GCE node
I20260808 04:58:46.644665 14991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:46.644892 14991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:46.646034 14991 hybrid_clock.cc:648] HybridClock initialized: now 1786165126646022 us; error 30 us; skew 500 ppm
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:46.648458 14991 init.cc:377] Logged in from keytab as oryx/127.3.174.131@KRBTEST.COM (short username oryx)
I20260808 04:58:46.648929 14991 webserver.cc:460] Webserver started at http://127.3.174.131:35851/ using document root <none> and password file <none>
I20260808 04:58:46.649128 14991 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:46.649221 14991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:46.649411 14991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:46.650524 14991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "6cbd6fbf05d14b67bb4c7601e063004e"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "27ba1324fcb3db947af781584964c43e"
server_key_iv: "5b0cb952d6cad444b2d1c6c44e3c1eb5"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.650853 14991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "6cbd6fbf05d14b67bb4c7601e063004e"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "27ba1324fcb3db947af781584964c43e"
server_key_iv: "5b0cb952d6cad444b2d1c6c44e3c1eb5"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.652177 14991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:46.652940 15022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.653103 14991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:46.653190 14991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "6cbd6fbf05d14b67bb4c7601e063004e"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "27ba1324fcb3db947af781584964c43e"
server_key_iv: "5b0cb952d6cad444b2d1c6c44e3c1eb5"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.653321 14991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:46.661055 14991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:46.662822 14991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:46.662954 14991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:46.663177 14991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:46.663522 14991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:46.663566 14991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.663602 14991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:46.663626 14991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.669350 14991 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:43685
I20260808 04:58:46.669422 15219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:43685 every 8 connection(s)
I20260808 04:58:46.669713 14991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:46.678298 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.670393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:59727 (local address 127.3.174.190:35583)
0808 04:58:46.670443 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.670446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.670668 (+ 222us) server_negotiation.cc:407] Connection header received
0808 04:58:46.671279 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.671281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.671291 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.671342 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.672295 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.672795 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.673817 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.673993 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.674039 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.676009 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.676010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.676010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.676017 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.677141 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.677565 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.677566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.677566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.677615 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.677803 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.677803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.677803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.677897 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.677940 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.678144 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.678202 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 04:58:46.678488 15224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.670376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.131:59727)
0808 04:58:46.670540 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:46.670577 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:46.671168 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:46.671416 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:46.671420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:46.671990 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:46.672182 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.672188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.672866 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.672867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.673749 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.673752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.674047 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.674048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.674143 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.674565 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:46.674573 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:46.675921 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:46.677238 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.677245 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.677247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.677510 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.677663 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.677664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.677664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.677760 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.677938 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:46.677941 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:46.678053 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 04:58:46.678130 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 04:58:46.678282 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260808 04:58:46.678666 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 14991
I20260808 04:58:46.678761 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:46.678962 3770 external_mini_cluster.cc:1428] Setting key 0d90390ed699f1be50ddab72634eee14
I20260808 04:58:46.678979 15221 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35583
I20260808 04:58:46.679073 15221 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:46.679258 15221 heartbeater.cc:511] Master 127.3.174.190:35583 requested a full tablet report, sending...
I20260808 04:58:46.679646 14364 ts_manager.cc:194] Registered new tserver with Master: 6cbd6fbf05d14b67bb4c7601e063004e (127.3.174.131:43685)
I20260808 04:58:46.680013 14364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:59727
I20260808 04:58:46.690589 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.174.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.189.keytab.
Entry for principal oryx/127.3.174.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.189.keytab.
I20260808 04:58:46.716367 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.189:38099
--webserver_interface=127.3.174.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.189.keytab
--principal=oryx/127.3.174.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:35583,127.3.174.189:38099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:46.800596 15233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:46.800806 15233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:46.800838 15233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:46.802000 15233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:46.802040 15233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:46.802060 15233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:46.802075 15233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:46.802089 15233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:46.803516 15233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:35583,127.3.174.189:38099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.189.keytab
--principal=oryx/127.3.174.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.189:38099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.174.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
W20260808 04:58:46.803640 15233 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.3.174.190:35583,127.3.174.189:38099'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 04:58:46.803781 15233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:46.803977 15233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:46.808514 15246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:46.808519 15253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:46.808547 15248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:46.808653 15233 server_base.cc:1034] running on GCE node
I20260808 04:58:46.808892 15233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:46.809170 15233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:46.810321 15233 hybrid_clock.cc:648] HybridClock initialized: now 1786165126810311 us; error 37 us; skew 500 ppm
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:46.812867 15233 init.cc:377] Logged in from keytab as oryx/127.3.174.189@KRBTEST.COM (short username oryx)
I20260808 04:58:46.813344 15233 webserver.cc:460] Webserver started at http://127.3.174.189:36087/ using document root <none> and password file <none>
I20260808 04:58:46.813555 15233 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:46.813602 15233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:46.813745 15233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:46.814913 15233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data/instance:
uuid: "ff9ef9171b664a6980dc877479e811be"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "7102b408154619bf4b7b6996b0960292"
server_key_iv: "b984a9ee4c449e5827431b2fbc8b415a"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.815270 15233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal/instance:
uuid: "ff9ef9171b664a6980dc877479e811be"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "7102b408154619bf4b7b6996b0960292"
server_key_iv: "b984a9ee4c449e5827431b2fbc8b415a"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.816540 15233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:46.817348 15266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:46.817485 15233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:46.817570 15233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal
uuid: "ff9ef9171b664a6980dc877479e811be"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "7102b408154619bf4b7b6996b0960292"
server_key_iv: "b984a9ee4c449e5827431b2fbc8b415a"
server_key_version: "encryptionkey@0"
I20260808 04:58:46.817638 15233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:46.837754 15233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:46.839952 15233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:46.840091 15233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:46.844090 15233 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.189:38099
I20260808 04:58:46.844123 15368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.189:38099 every 8 connection(s)
I20260808 04:58:46.844489 15233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
I20260808 04:58:46.845577 15370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:46.847323 15370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:58:46.851879 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 15233
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.189@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:46.858057 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.848072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.3.174.190:35583)
0808 04:58:46.848131 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.848134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.848362 (+ 228us) server_negotiation.cc:407] Connection header received
0808 04:58:46.848876 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.848877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.848888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.848938 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.849625 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.850123 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.851286 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.851470 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.851519 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.854063 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.854064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.854065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.854074 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.856097 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.856497 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.856499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.856499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.856563 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.856934 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.856935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.856935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.857191 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.857396 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.857838 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.857964 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 04:58:46.858309 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.848045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56354)
0808 04:58:46.848233 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:46.848271 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:46.848785 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:46.849024 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:46.849027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:46.849339 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:46.849537 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.849542 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.850248 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.850249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.851209 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.851212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.851528 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.851528 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.851628 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.852124 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:46.852133 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:46.853974 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:46.856192 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.856198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.856200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.856420 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.856637 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.856638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.856638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.856820 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.857373 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:46.857380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:46.857555 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 04:58:46.857753 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 04:58:46.858013 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.174.189@KRBTEST.COM
I20260808 04:58:46.859274 15370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } has no permanent_uuid. Determining permanent_uuid...
I20260808 04:58:46.862690 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.852369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43380 (local address 127.3.174.189:38099)
0808 04:58:46.852530 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.852535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.852605 (+ 70us) server_negotiation.cc:407] Connection header received
0808 04:58:46.852681 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.852699 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.852740 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.852906 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.853630 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.854612 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.855656 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.856291 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.856351 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.859574 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.859581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.859581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.859586 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.861087 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.861473 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.861476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.861477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.861541 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.861804 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.861805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.861805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.862017 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.862255 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.862322 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.862555 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.189@KRBTEST.COM for oryx/127.3.174.189@KRBTEST.COM
I20260808 04:58:46.869670 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.859560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.3.174.189:38099)
0808 04:58:46.859693 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.859695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.859705 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:46.859775 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.859775 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.859784 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.859858 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.860102 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.860608 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.861313 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.861532 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.861584 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.865400 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.865401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.865401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.865418 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.867889 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.868341 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.868342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.868342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.868401 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.868748 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.868748 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.868749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.868954 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.869044 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.869251 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.869502 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260808 04:58:46.869807 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.859515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.0.0.1:43394)
0808 04:58:46.859670 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:46.859679 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:46.859718 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:46.859860 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:46.859861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:46.859992 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:46.860042 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.860043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.860707 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.860708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.861220 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.861222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.861634 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.861634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.861704 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.862258 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:46.862263 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:46.865251 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:46.867985 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.867986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.867986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.868174 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.868474 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.868474 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.868475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.868656 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.869037 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:46.869037 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:46.869105 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 04:58:46.869215 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 04:58:46.869599 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":122}
I20260808 04:58:46.873423 15370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: Bootstrap starting.
I20260808 04:58:46.874455 15370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:46.874742 15370 log.cc:824] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:46.875530 15370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: No bootstrap required, opened a new log
I20260808 04:58:46.878027 15370 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } }
I20260808 04:58:46.878641 15370 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:46.878743 15370 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff9ef9171b664a6980dc877479e811be, State: Initialized, Role: FOLLOWER
I20260808 04:58:46.878994 15370 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } }
I20260808 04:58:46.879640 15370 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:46.879530 15387 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } } }
I20260808 04:58:46.879772 15387 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: FOLLOWER
W20260808 04:58:46.883419 15407 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:58:46.883517 15407 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:58:46.888216 15233 master_runner.cc:428] Detected that this master ff9ef9171b664a6980dc877479e811be is joining an existing cluster
I20260808 04:58:46.888444 15233 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.174.189:38099
I20260808 04:58:46.912074 15387 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:46.912689 15387 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } }
I20260808 04:58:46.913796 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.893398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.3.174.189:38099)
0808 04:58:46.893683 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.893685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.894350 (+ 665us) server_negotiation.cc:407] Connection header received
0808 04:58:46.894693 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.894701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.894757 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.894884 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.896368 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.898026 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.899946 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.901170 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.901592 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.904982 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.904984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.904985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.905022 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.909260 (+ 4238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.910270 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.910270 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.910271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.910401 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.910870 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.910871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.910878 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.911139 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.911317 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.912805 (+ 1488us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.913482 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":266,"server-negotiator.queue_time_us":199}
I20260808 04:58:46.914660 15420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.893267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.0.0.1:43406)
0808 04:58:46.894122 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:46.894178 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:46.894356 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:46.895082 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:46.895083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:46.895978 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:46.896085 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.896088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.898283 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.898284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.899788 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.899796 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.901641 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.901651 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.902039 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.903196 (+ 1157us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:46.903229 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:46.904686 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:46.909560 (+ 4874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.909561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.909561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.910056 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.910629 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.910630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.910630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.910759 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.911356 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:46.911365 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:46.911467 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 04:58:46.912073 (+ 606us) client_negotiation.cc:240] Negotiation successful
0808 04:58:46.913006 (+ 933us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":244,"threads_started":1}
I20260808 04:58:46.915768 14380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff9ef9171b664a6980dc877479e811be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8917e6eeb4234234b3b255985ff749e9" is_pre_election: true
I20260808 04:58:46.916854 15273 leader_election.cc:302] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ff9ef9171b664a6980dc877479e811be; no voters: 8917e6eeb4234234b3b255985ff749e9
I20260808 04:58:46.916942 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.893268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56368)
0808 04:58:46.894260 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:46.894272 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:46.894447 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:46.895107 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:46.895109 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:46.896087 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:46.896244 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.896247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.899976 (+ 3729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.899980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.902536 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.902544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.903815 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.903816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.903960 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.904670 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:46.904682 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:46.906884 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:46.913117 (+ 6233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.913118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.913119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.914009 (+ 890us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.914654 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.914655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.914655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.914860 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.916058 (+ 1198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:46.916066 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:46.916256 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 04:58:46.916539 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 04:58:46.916639 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":485,"thread_start_us":401,"threads_started":1}
I20260808 04:58:46.917263 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.893520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.3.174.190:35583)
0808 04:58:46.893663 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.893681 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.894429 (+ 748us) server_negotiation.cc:407] Connection header received
0808 04:58:46.894793 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.894802 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.894885 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.895053 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.896740 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.899833 (+ 3093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.903089 (+ 3256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.903562 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.903798 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.906999 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.907001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.907001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.907009 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.912802 (+ 5793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.914269 (+ 1467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.914271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.914271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.914435 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.915084 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.915086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.915086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.915658 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.915919 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.917075 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.917159 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 04:58:46.913904 15387 leader_election.cc:288] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8917e6eeb4234234b3b255985ff749e9 (127.3.174.190:35583)
I20260808 04:58:46.919088 15387 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
WARNING: no policy specified for oryx/127.3.174.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.188@KRBTEST.COM" created.
Aug 08 04:58:46 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.190@KRBTEST.COM for oryx/127.3.174.189@KRBTEST.COM
I20260808 04:58:46.961148 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.931354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.0.0.1:43418)
0808 04:58:46.932200 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:46.932321 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:46.932746 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:46.933922 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:46.933938 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:46.936011 (+ 2073us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:46.936490 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.936523 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.940307 (+ 3784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.940309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.942940 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:46.942944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.944923 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.944932 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:46.945274 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.946808 (+ 1534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:46.946816 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:46.955420 (+ 8604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:46.957830 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.957841 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.957844 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.958097 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.958292 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:46.958293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:46.958293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:46.958364 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:46.959049 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:46.959058 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:46.959174 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 04:58:46.959446 (+ 272us) client_negotiation.cc:240] Negotiation successful
0808 04:58:46.960760 (+ 1314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":414,"threads_started":1}
I20260808 04:58:46.961468 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:46.931597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.3.174.189:38099)
0808 04:58:46.931867 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:46.931885 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:46.932607 (+ 722us) server_negotiation.cc:407] Connection header received
0808 04:58:46.933257 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:46.933259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:46.933376 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:46.933680 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:46.937048 (+ 3368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.939509 (+ 2461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.943371 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:46.944678 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:46.944960 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:46.955603 (+ 10643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:46.955621 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:46.955622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:46.955689 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:46.957739 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.958167 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.958168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.958168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.958229 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:46.958421 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:46.958422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:46.958422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:46.958579 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:46.958628 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:46.960868 (+ 2240us) server_negotiation.cc:299] Negotiation successful
0808 04:58:46.961108 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260808 04:58:46.962561 14364 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.174.189:38099: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ff9ef9171b664a6980dc877479e811be"
member_type: NON_VOTER
last_known_addr {
host: "127.3.174.189"
port: 38099
}
attrs {
promote: true
}
}
dest_uuid: "8917e6eeb4234234b3b255985ff749e9"
cas_config_opid_index: -1
I20260808 04:58:46.963428 14364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } }
I20260808 04:58:46.966079 15428 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ff9ef9171b664a6980dc877479e811be (127.3.174.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:46.967134 15313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:46.968900 15313 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 FOLLOWER]: Refusing update from remote peer 8917e6eeb4234234b3b255985ff749e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 04:58:46.969462 15427 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:46.970500 15428 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.174.189:38099
I20260808 04:58:46.971202 15233 master.cc:537] Master@127.3.174.189:38099 shutting down...
I20260808 04:58:46.976167 15427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:46.976378 15427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:46.977952 15313 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ff9ef9171b664a6980dc877479e811be (127.3.174.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:46.982796 15422 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8917e6eeb4234234b3b255985ff749e9. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:46.983031 15422 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: LEARNER
I20260808 04:58:46.983011 15435 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:46.983180 15435 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: LEARNER
I20260808 04:58:46.983224 15387 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:46.983350 15387 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: LEARNER
I20260808 04:58:46.984881 15430 mvcc.cc:205] Tried to move back new op lower bound from 7316132356912726016 to 7316132356762738688. Current Snapshot: MvccSnapshot[applied={T|T < 7316132356780167168}]
I20260808 04:58:46.984961 15233 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Raft consensus shutting down.
I20260808 04:58:46.985159 15233 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Raft consensus is shut down!
I20260808 04:58:46.985285 15233 tablet_replica.cc:330] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.188.keytab.
Entry for principal oryx/127.3.174.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.188.keytab.
I20260808 04:58:46.995827 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.188:37103
--webserver_interface=127.3.174.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.188.keytab
--principal=oryx/127.3.174.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:35583,127.3.174.189:38099,127.3.174.188:37103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:58:47.004949 15233 master.cc:552] Master@127.3.174.189:38099 shutdown complete.
I20260808 04:58:47.005249 15233 master_runner.cc:305] Clearing existing system tablet
I20260808 04:58:47.006536 15233 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:47.006707 15233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:47.010195 15233 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 04:58:47.012099 15446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.012599 15233 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:47.012723 15233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 04:58:47.012771 15233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal
uuid: "ff9ef9171b664a6980dc877479e811be"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "7102b408154619bf4b7b6996b0960292"
server_key_iv: "b984a9ee4c449e5827431b2fbc8b415a"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.016201 15233 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:47.020812 15233 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 04:58:47.020906 15233 log.cc:1196] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 04:58:47.021145 15233 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: Deleting consensus metadata
I20260808 04:58:47.021318 15233 master_runner.cc:315] Copying system tablet from 127.3.174.190:35583
I20260808 04:58:47.022810 15233 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: tablet copy: Beginning tablet copy session from remote peer at address 127.3.174.190:35583
I20260808 04:58:47.054836 15454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.024448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56372)
0808 04:58:47.025336 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.025401 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.025753 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.026729 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.026745 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.028674 (+ 1929us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.029237 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.029249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.032715 (+ 3466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.032724 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.035041 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.035045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.035647 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.035648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.035721 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.037645 (+ 1924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.037690 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.040471 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.051354 (+ 10883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.051364 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.051372 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.052906 (+ 1534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.054066 (+ 1160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.054075 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.054075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.054164 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.054460 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.054460 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.054516 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.054574 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.054699 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":332,"threads_started":1}
I20260808 04:58:47.054842 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.024705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.3.174.190:35583)
0808 04:58:47.024970 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.024981 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.025652 (+ 671us) server_negotiation.cc:407] Connection header received
0808 04:58:47.026323 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.026325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.026387 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.026630 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.029776 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.032513 (+ 2737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.035142 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.035560 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.035637 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.040843 (+ 5206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.040867 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.040875 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.040959 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.050965 (+ 10006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.053476 (+ 2511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.053486 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.053494 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.053766 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.054248 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.054249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.054249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.054402 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.054467 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.054635 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.054736 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260808 04:58:47.058120 14404 tablet_copy_service.cc:140] P 8917e6eeb4234234b3b255985ff749e9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ff9ef9171b664a6980dc877479e811be ({username='oryx', principal='oryx/127.3.174.189@KRBTEST.COM'} at 127.0.0.1:56372)
I20260808 04:58:47.058530 14404 tablet_copy_service.cc:161] P 8917e6eeb4234234b3b255985ff749e9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ff9ef9171b664a6980dc877479e811be at {username='oryx', principal='oryx/127.3.174.189@KRBTEST.COM'} at 127.0.0.1:56372: session id = ff9ef9171b664a6980dc877479e811be-00000000000000000000000000000000
I20260808 04:58:47.062645 14404 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 04:58:47.065281 15233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 04:58:47.068789 15233 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: tablet copy: Starting download of 0 data blocks...
I20260808 04:58:47.068974 15233 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: tablet copy: Starting download of 1 WAL segments...
I20260808 04:58:47.070169 15233 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 04:58:47.071213 14404 tablet_copy_service.cc:342] P 8917e6eeb4234234b3b255985ff749e9: Request end of tablet copy session ff9ef9171b664a6980dc877479e811be-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.174.189@KRBTEST.COM'} at 127.0.0.1:56372
I20260808 04:58:47.071281 14404 tablet_copy_service.cc:433] P 8917e6eeb4234234b3b255985ff749e9: ending tablet copy session ff9ef9171b664a6980dc877479e811be-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ff9ef9171b664a6980dc877479e811be
I20260808 04:58:47.074631 15233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:47.075875 15456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.076220 15462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:47.076375 15233 server_base.cc:1034] running on GCE node
W20260808 04:58:47.077402 15457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:47.077625 15233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:47.077806 15233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:47.078933 15233 hybrid_clock.cc:648] HybridClock initialized: now 1786165127078924 us; error 27 us; skew 500 ppm
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:47.080542 15233 init.cc:377] Logged in from keytab as oryx/127.3.174.189@KRBTEST.COM (short username oryx)
I20260808 04:58:47.080869 15233 webserver.cc:460] Webserver started at http://127.3.174.189:39329/ using document root <none> and password file <none>
I20260808 04:58:47.080996 15233 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:47.081034 15233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:47.081810 15233 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:47.082402 15471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.082564 15233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:47.082623 15233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal
uuid: "ff9ef9171b664a6980dc877479e811be"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "7102b408154619bf4b7b6996b0960292"
server_key_iv: "b984a9ee4c449e5827431b2fbc8b415a"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.082693 15233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:47.098133 15233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:47.099980 15233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:47.104387 15233 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.189:38099
I20260808 04:58:47.104913 15233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
I20260808 04:58:47.105316 15572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.189:38099 every 8 connection(s)
I20260808 04:58:47.106215 15573 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:58:47.106755 15573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: Bootstrap starting.
I20260808 04:58:47.109690 15573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:58:47.110096 15573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: Bootstrap complete.
I20260808 04:58:47.110390 15573 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } }
I20260808 04:58:47.110560 15573 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9ef9171b664a6980dc877479e811be, State: Initialized, Role: LEARNER
I20260808 04:58:47.110647 15573 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } }
I20260808 04:58:47.110924 15573 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:47.110953 15579 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:47.111064 15579 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: LEARNER
I20260808 04:58:47.112762 15597 catalog_manager.cc:1259] Loaded cluster ID: 10dd19f8c9d84032a41440f7466a608b
I20260808 04:58:47.112814 15597 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: loading cluster ID for follower catalog manager: success
I20260808 04:58:47.113933 15597 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: acquiring CA information for follower catalog manager: success
I20260808 04:58:47.114254 15597 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 04:58:47.147809 15440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:47.148015 15440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:47.148046 15440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:47.149277 15440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:47.149348 15440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:47.149367 15440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:47.149382 15440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:47.149407 15440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:47.151031 15440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:35583,127.3.174.189:38099,127.3.174.188:37103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.188.keytab
--principal=oryx/127.3.174.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.188:37103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.174.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:47.151320 15440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:47.151526 15440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:47.156334 15615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:47.156760 15440 server_base.cc:1034] running on GCE node
W20260808 04:58:47.156332 15616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.156934 15618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:47.157276 15440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:47.157608 15440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:47.158754 15440 hybrid_clock.cc:648] HybridClock initialized: now 1786165127158741 us; error 32 us; skew 500 ppm
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:47.161248 15440 init.cc:377] Logged in from keytab as oryx/127.3.174.188@KRBTEST.COM (short username oryx)
I20260808 04:58:47.161859 15440 webserver.cc:460] Webserver started at http://127.3.174.188:41515/ using document root <none> and password file <none>
I20260808 04:58:47.162066 15440 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:47.162114 15440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:47.162264 15440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:47.163503 15440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data/instance:
uuid: "36bd4278aaa94063923d5e6ffddef9e4"
format_stamp: "Formatted at 2026-08-08 04:58:47 on dist-test-slave-gjw7"
server_key: "d7a16e8c4f9f502e9452f4a5292442a9"
server_key_iv: "9175fb428715fde8cb03a2334802ba87"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.163897 15440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal/instance:
uuid: "36bd4278aaa94063923d5e6ffddef9e4"
format_stamp: "Formatted at 2026-08-08 04:58:47 on dist-test-slave-gjw7"
server_key: "d7a16e8c4f9f502e9452f4a5292442a9"
server_key_iv: "9175fb428715fde8cb03a2334802ba87"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.165390 15440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 04:58:47.166134 15634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.166301 15440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:47.166390 15440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal
uuid: "36bd4278aaa94063923d5e6ffddef9e4"
format_stamp: "Formatted at 2026-08-08 04:58:47 on dist-test-slave-gjw7"
server_key: "d7a16e8c4f9f502e9452f4a5292442a9"
server_key_iv: "9175fb428715fde8cb03a2334802ba87"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.166456 15440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:47.178972 15440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:47.181115 15440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:47.181257 15440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:47.188644 15440 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.188:37103
I20260808 04:58:47.188705 15729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.188:37103 every 8 connection(s)
I20260808 04:58:47.189114 15440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
I20260808 04:58:47.190392 15732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:47.191048 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 15440
I20260808 04:58:47.193408 15732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.174.188@KRBTEST.COM
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.188@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:47.201961 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.191770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.3.174.188:37103)
0808 04:58:47.191969 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.191978 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.192004 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:58:47.192464 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.192472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.192546 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.192669 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.193352 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.194712 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.195433 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.195988 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.196097 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.198625 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.198634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.198634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.198641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.200480 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.200798 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.200802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.200802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.200884 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.201052 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.201053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.201053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.201281 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.201453 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.201566 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.201684 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
I20260808 04:58:47.202018 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.194326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56386 (local address 127.3.174.190:35583)
0808 04:58:47.194381 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.194383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.194581 (+ 198us) server_negotiation.cc:407] Connection header received
0808 04:58:47.194632 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.194634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.194645 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.194714 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.195448 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.196000 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.197021 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.197203 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.197256 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.199632 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.199633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.199633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.199640 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.200888 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.201285 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.201305 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.201305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.201365 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.201531 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.201532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.201532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.201636 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.201691 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.201855 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.201932 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 04:58:47.202180 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.194176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56386)
0808 04:58:47.194477 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.194509 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.194578 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.194757 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.194760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.195174 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.195362 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.195364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.196176 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.196178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.196947 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.196950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.197392 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.197393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.197490 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.198076 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.198086 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.199557 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.200972 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.200979 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.200981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.201161 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.201422 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.201422 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.201423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.201482 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.201691 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.201698 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.201747 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.201823 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.201932 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":195,"threads_started":1}
I20260808 04:58:47.202704 15732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.188@KRBTEST.COM for oryx/127.3.174.189@KRBTEST.COM
I20260808 04:58:47.209205 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.202962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.0.0.1:43428)
0808 04:58:47.203003 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.203015 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.203057 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.203289 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.203290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.203390 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.203474 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.203476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.204282 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.204283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.204774 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.204775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.205144 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.205145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.205277 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.205683 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.205687 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.206941 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.208489 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.208490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.208490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.208638 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.208805 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.208805 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.208805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.208847 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.209015 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.209015 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.209045 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.209084 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.209146 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 04:58:47.209227 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.203046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43428 (local address 127.3.174.189:38099)
0808 04:58:47.203161 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.203163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.203169 (+ 6us) server_negotiation.cc:407] Connection header received
0808 04:58:47.203190 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.203191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.203235 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.203286 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.203565 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.204189 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.204860 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.205063 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.205124 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.207016 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.207018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.207018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.207025 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.208419 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.208691 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.208692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.208692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.208741 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.208885 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.208885 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.208885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.208978 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.209019 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.209104 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.209163 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260808 04:58:47.209685 15732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.188@KRBTEST.COM for oryx/127.3.174.188@KRBTEST.COM
I20260808 04:58:47.215682 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.210075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37248 (local address 127.3.174.188:37103)
0808 04:58:47.210100 (+ 25us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.210101 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.210105 (+ 4us) server_negotiation.cc:407] Connection header received
0808 04:58:47.210117 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.210117 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.210122 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.210152 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.210357 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.210915 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.211647 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.211949 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.212002 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.213591 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.213592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.213592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.213598 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.214758 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.215030 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.215031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.215031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.215072 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.215282 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.215282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.215282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.215360 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.215399 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.215566 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.215627 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 04:58:47.215780 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.209946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.0.0.1:37248)
0808 04:58:47.209986 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.209996 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.210030 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.210176 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.210177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.210284 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.210310 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.210311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.210974 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.210975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.211594 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.211596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.212009 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.212009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.212048 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.212425 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.212430 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.213506 (+ 1076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.214828 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.214829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.214829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.214976 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.215136 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.215136 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.215137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.215236 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.215441 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.215442 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.215483 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.215550 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.215677 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":46}
I20260808 04:58:47.217621 15732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Bootstrap starting.
I20260808 04:58:47.218958 15732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:47.219429 15732 log.cc:824] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:47.220327 15732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: No bootstrap required, opened a new log
I20260808 04:58:47.223574 15732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } }
I20260808 04:58:47.223703 15732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:47.223732 15732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36bd4278aaa94063923d5e6ffddef9e4, State: Initialized, Role: FOLLOWER
I20260808 04:58:47.223860 15732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } }
I20260808 04:58:47.224357 15732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:47.224337 15742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } } }
I20260808 04:58:47.224409 15742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 04:58:47.227597 15763 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:58:47.227648 15763 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:58:47.229889 15440 master_runner.cc:428] Detected that this master 36bd4278aaa94063923d5e6ffddef9e4 is joining an existing cluster
I20260808 04:58:47.229940 15440 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.174.188:37103
I20260808 04:58:47.239507 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.232153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.3.174.188:37103)
0808 04:58:47.232186 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.232188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.232474 (+ 286us) server_negotiation.cc:407] Connection header received
0808 04:58:47.232559 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.232560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.232567 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.232591 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.233227 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.233913 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.235233 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.235500 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.235537 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.237093 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.237094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.237094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.237100 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.238415 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.238783 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.238783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.238784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.238827 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.239003 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.239003 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.239004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.239088 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.239135 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.239383 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.239448 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 04:58:47.239949 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.232015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.0.0.1:37254)
0808 04:58:47.232411 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.232422 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.232498 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.232649 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.232650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.232954 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.233127 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.233130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.233998 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.233999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.235125 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.235128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.235723 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.235724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.235889 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.236354 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.236359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.236932 (+ 573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.238492 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.238493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.238494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.238714 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.238879 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.238880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.238880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.238952 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.239149 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.239150 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.239285 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.239363 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.239528 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":55,"thread_start_us":74,"threads_started":1}
I20260808 04:58:47.251472 15742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:47.251603 15742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } }
I20260808 04:58:47.251971 15742 leader_election.cc:288] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8917e6eeb4234234b3b255985ff749e9 (127.3.174.190:35583), ff9ef9171b664a6980dc877479e811be (127.3.174.189:38099)
I20260808 04:58:47.252218 14380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36bd4278aaa94063923d5e6ffddef9e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8917e6eeb4234234b3b255985ff749e9" is_pre_election: true
I20260808 04:58:47.252305 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36bd4278aaa94063923d5e6ffddef9e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff9ef9171b664a6980dc877479e811be" is_pre_election: true
I20260808 04:58:47.252429 15534 raft_consensus.cc:1799] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Handling vote request from an unknown peer 36bd4278aaa94063923d5e6ffddef9e4
I20260808 04:58:47.252478 15534 raft_consensus.cc:2396] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 36bd4278aaa94063923d5e6ffddef9e4 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260808 04:58:47.252739 15641 leader_election.cc:302] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36bd4278aaa94063923d5e6ffddef9e4; no voters: 8917e6eeb4234234b3b255985ff749e9, ff9ef9171b664a6980dc877479e811be
I20260808 04:58:47.252995 15742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:47.254405 15742 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 04:58:47.255093 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 14480
I20260808 04:58:47.274336 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.231491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.3.174.190:35583)
0808 04:58:47.231535 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.231537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.232286 (+ 749us) server_negotiation.cc:407] Connection header received
0808 04:58:47.232396 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.232397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.232411 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.232442 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.232922 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.233741 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.235214 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.235517 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.235555 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.271245 (+ 35690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.271247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.271248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.271257 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.273170 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.273544 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.273545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.273545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.273594 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.273792 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.273793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.273794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.273920 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.273963 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.274156 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.274225 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 04:58:47.274336 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.231696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.3.174.189:38099)
0808 04:58:47.231764 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.231766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.232060 (+ 294us) server_negotiation.cc:407] Connection header received
0808 04:58:47.232116 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.232117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.232134 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.232191 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.233028 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.233765 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.235214 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.235533 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.235566 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.271245 (+ 35679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.271247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.271248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.271268 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.272914 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.273375 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.273377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.273377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.273432 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.273628 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.273629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.273629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.273774 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.273816 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.274115 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.274217 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 04:58:47.274760 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.231462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.0.0.1:43432)
0808 04:58:47.231994 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.232005 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.232064 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.232214 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.232215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.232599 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.232945 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.232947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.233843 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.233844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.235113 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.235116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.235708 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.235709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.235868 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.270393 (+ 34525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.270401 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.271092 (+ 691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.273019 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.273021 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.273021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.273227 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.273501 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.273502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.273502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.273559 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.273864 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.273864 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.273998 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.274468 (+ 470us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.274595 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":35,"thread_start_us":394,"threads_started":1}
I20260808 04:58:47.275313 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.231584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56388)
0808 04:58:47.232177 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.232186 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.232316 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.232505 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.232506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.232764 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.232818 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.232820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.233833 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.233834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.235130 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.235131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.235711 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.235711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.235817 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.270431 (+ 34614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.270436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.271105 (+ 669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.273260 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.273261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.273261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.273465 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.273662 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.273663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.273663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.273712 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.274011 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.274012 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.274048 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.275045 (+ 997us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.275155 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":44,"thread_start_us":380,"threads_started":1}
I20260808 04:58:47.276561 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:35607
--local_ip_for_outbound_sockets=127.3.174.129
--tserver_master_addrs=127.3.174.190:35583
--webserver_port=38211
--webserver_interface=127.3.174.129
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab
--principal=oryx/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.174.190:35583,127.3.174.189:38099
--tserver_master_addrs=127.3.174.190:35583,127.3.174.189:38099,127.3.174.188:37103 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165126, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.190@KRBTEST.COM for oryx/127.3.174.188@KRBTEST.COM
I20260808 04:58:47.284206 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.277619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37268 (local address 127.3.174.188:37103)
0808 04:58:47.277674 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.277676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.277683 (+ 7us) server_negotiation.cc:407] Connection header received
0808 04:58:47.277718 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.277719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.277730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.277784 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.278120 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.278868 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.279823 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.280007 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.280070 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.281820 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.281821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.281821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.281828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.283120 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.283462 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.283463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.283464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.283517 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.283716 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.283717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.283717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.283855 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.283911 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.284048 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.284118 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 04:58:47.284219 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.277529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.0.0.1:37268)
0808 04:58:47.277579 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.277590 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.277633 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.277799 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.277800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.277976 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.278032 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.278033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.278958 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.278959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.279740 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.279744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.280087 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.280089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.280161 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.280491 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.280496 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.281720 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.283198 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.283199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.283199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.283404 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.283590 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.283591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.283591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.283650 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.283913 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.283914 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.283958 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.284018 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.284118 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 04:58:47.284636 14364 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.174.188:37103: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4"
member_type: NON_VOTER
last_known_addr {
host: "127.3.174.188"
port: 37103
}
attrs {
promote: true
}
}
dest_uuid: "8917e6eeb4234234b3b255985ff749e9"
cas_config_opid_index: 5
I20260808 04:58:47.284888 14364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } }
I20260808 04:58:47.285244 15427 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 36bd4278aaa94063923d5e6ffddef9e4 (127.3.174.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.285686 15686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 FOLLOWER]: Refusing update from remote peer 8917e6eeb4234234b3b255985ff749e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 04:58:47.286460 15427 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.174.188:37103
I20260808 04:58:47.286494 15780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:47.286545 15427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.286605 15427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:47.286854 15440 master.cc:537] Master@127.3.174.188:37103 shutting down...
I20260808 04:58:47.287885 15686 raft_consensus.cc:858] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } }
W20260808 04:58:47.287979 15686 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } }
I20260808 04:58:47.288640 15686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ff9ef9171b664a6980dc877479e811be (127.3.174.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:47.289942 15742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8917e6eeb4234234b3b255985ff749e9. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:47.290025 15742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 04:58:47.290061 15742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:47.290104 15742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 04:58:47.290135 15742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } }
I20260808 04:58:47.290181 15742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 04:58:47.292616 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.285900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.0.0.1:43444)
0808 04:58:47.289768 (+ 3868us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.289780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.289861 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.290084 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.290085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.290087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:47.290123 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.290125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.290802 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.290802 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.291516 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.291517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.292356 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.292357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.292422 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.292459 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.292506 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.292543 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3837}
I20260808 04:58:47.292879 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.287420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43444 (local address 127.3.174.189:38099)
0808 04:58:47.287459 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.287461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.289831 (+ 2370us) server_negotiation.cc:407] Connection header received
0808 04:58:47.289935 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.289936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.289960 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.289999 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:47.290202 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.290718 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.291607 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.292155 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.292225 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.292317 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.292769 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.292814 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 04:58:47.293078 15534 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Refusing update from remote peer 8917e6eeb4234234b3b255985ff749e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 04:58:47.293321 15781 mvcc.cc:205] Tried to move back new op lower bound from 7316132356912726016 to 7316132356762738688. Current Snapshot: MvccSnapshot[applied={T|T < 7316132356780167168}]
I20260808 04:58:47.293402 15428 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 04:58:47.293429 15440 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260808 04:58:47.293524 15440 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260808 04:58:47.293509 15428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.293562 15428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:47.293565 15440 tablet_replica.cc:330] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: stopping tablet replica
I20260808 04:58:47.294062 15534 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 36bd4278aaa94063923d5e6ffddef9e4 (127.3.174.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.295174 15579 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8917e6eeb4234234b3b255985ff749e9. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.295261 15579 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: LEARNER
W20260808 04:58:47.295243 14341 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260808 04:58:47.295297 15579 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: NON_VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: true } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.295336 15579 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: LEARNER
I20260808 04:58:47.295637 15427 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: attempting to promote NON_VOTER ff9ef9171b664a6980dc877479e811be to VOTER
I20260808 04:58:47.295825 15427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } }
I20260808 04:58:47.296367 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.296179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.0.0.1:37282)
0808 04:58:47.296303 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.296344 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:37103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260808 04:58:47.296545 15534 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 LEARNER]: Refusing update from remote peer 8917e6eeb4234234b3b255985ff749e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 04:58:47.296715 15428 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 04:58:47.296936 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.296860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.0.0.1:37284)
0808 04:58:47.296888 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.296913 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:37103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260808 04:58:47.297425 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.297202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.0.0.1:37296)
0808 04:58:47.297237 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.297266 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:37103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 04:58:47.297520 14341 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 -> Peer 36bd4278aaa94063923d5e6ffddef9e4 (127.3.174.188:37103): Couldn't send request to peer 36bd4278aaa94063923d5e6ffddef9e4. Status: Network error: Client connection negotiation failed: client connection to 127.3.174.188:37103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 04:58:47.297684 15780 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ff9ef9171b664a6980dc877479e811be (127.3.174.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.297828 15534 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ff9ef9171b664a6980dc877479e811be (127.3.174.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.298647 15579 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.298722 15579 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:58:47.298935 15428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.299011 15428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:47.299088 15428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.299137 15428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:47.312619 15440 master.cc:552] Master@127.3.174.188:37103 shutdown complete.
I20260808 04:58:47.312670 15440 master_runner.cc:305] Clearing existing system tablet
I20260808 04:58:47.312813 15440 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:47.312860 15440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:47.313428 15440 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.313872 15807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.314009 15440 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:47.314061 15440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.314085 15440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal
uuid: "36bd4278aaa94063923d5e6ffddef9e4"
format_stamp: "Formatted at 2026-08-08 04:58:47 on dist-test-slave-gjw7"
server_key: "d7a16e8c4f9f502e9452f4a5292442a9"
server_key_iv: "9175fb428715fde8cb03a2334802ba87"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.314673 15440 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:47.315969 15440 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 04:58:47.316032 15440 log.cc:1196] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 04:58:47.316244 15440 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Deleting consensus metadata
I20260808 04:58:47.316414 15440 master_runner.cc:315] Copying system tablet from 127.3.174.190:35583
I20260808 04:58:47.316960 15440 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: tablet copy: Beginning tablet copy session from remote peer at address 127.3.174.190:35583
I20260808 04:58:47.322526 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.317342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.3.174.190:35583)
0808 04:58:47.317386 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.317388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.317452 (+ 64us) server_negotiation.cc:407] Connection header received
0808 04:58:47.317515 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.317516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.317527 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.317573 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.317870 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.318362 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.319025 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.319220 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.319282 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.320143 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.320145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.320145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.320153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.321469 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.321784 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.321786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.321786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.321845 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.322036 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.322036 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.322037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.322166 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.322224 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.322359 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.322432 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:58:47.322530 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.317241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56394)
0808 04:58:47.317401 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.317410 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.317467 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.317577 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.317578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.317749 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.317798 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.317800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.318430 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.318431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.318954 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.318955 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.319280 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.319281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.319330 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.319748 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.319762 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.320059 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.321547 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.321548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.321548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.321722 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.321911 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.321911 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.321912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.321972 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.322218 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.322219 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.322266 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.322328 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.322434 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
I20260808 04:58:47.323119 14404 tablet_copy_service.cc:140] P 8917e6eeb4234234b3b255985ff749e9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 36bd4278aaa94063923d5e6ffddef9e4 ({username='oryx', principal='oryx/127.3.174.188@KRBTEST.COM'} at 127.0.0.1:56394)
I20260808 04:58:47.323177 14404 tablet_copy_service.cc:161] P 8917e6eeb4234234b3b255985ff749e9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 36bd4278aaa94063923d5e6ffddef9e4 at {username='oryx', principal='oryx/127.3.174.188@KRBTEST.COM'} at 127.0.0.1:56394: session id = 36bd4278aaa94063923d5e6ffddef9e4-00000000000000000000000000000000
I20260808 04:58:47.323647 14404 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 04:58:47.324079 15440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 04:58:47.325165 15440 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: tablet copy: Starting download of 0 data blocks...
I20260808 04:58:47.325358 15440 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: tablet copy: Starting download of 1 WAL segments...
I20260808 04:58:47.326050 15440 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 04:58:47.326794 14404 tablet_copy_service.cc:342] P 8917e6eeb4234234b3b255985ff749e9: Request end of tablet copy session 36bd4278aaa94063923d5e6ffddef9e4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.174.188@KRBTEST.COM'} at 127.0.0.1:56394
I20260808 04:58:47.326846 14404 tablet_copy_service.cc:433] P 8917e6eeb4234234b3b255985ff749e9: ending tablet copy session 36bd4278aaa94063923d5e6ffddef9e4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 36bd4278aaa94063923d5e6ffddef9e4
I20260808 04:58:47.328954 15440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:47.330024 15819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.330144 15821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.330327 15824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:47.336408 15440 server_base.cc:1034] running on GCE node
I20260808 04:58:47.336544 15440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:47.336679 15440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:47.337800 15440 hybrid_clock.cc:648] HybridClock initialized: now 1786165127337792 us; error 25 us; skew 500 ppm
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:47.339051 15440 init.cc:377] Logged in from keytab as oryx/127.3.174.188@KRBTEST.COM (short username oryx)
I20260808 04:58:47.339303 15440 webserver.cc:460] Webserver started at http://127.3.174.188:32923/ using document root <none> and password file <none>
I20260808 04:58:47.339416 15440 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:47.339448 15440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:47.340025 15440 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.340521 15836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.340616 15440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.340652 15440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal
uuid: "36bd4278aaa94063923d5e6ffddef9e4"
format_stamp: "Formatted at 2026-08-08 04:58:47 on dist-test-slave-gjw7"
server_key: "d7a16e8c4f9f502e9452f4a5292442a9"
server_key_iv: "9175fb428715fde8cb03a2334802ba87"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.340708 15440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:47.351720 15440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:47.353328 15440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:47.356825 15440 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.188:37103
I20260808 04:58:47.357163 15440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
I20260808 04:58:47.357949 15932 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:58:47.358342 15932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Bootstrap starting.
I20260808 04:58:47.359253 15931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.188:37103 every 8 connection(s)
I20260808 04:58:47.360188 15932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:58:47.360387 15932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Bootstrap complete.
I20260808 04:58:47.360569 15932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } }
I20260808 04:58:47.360639 15932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 36bd4278aaa94063923d5e6ffddef9e4, State: Initialized, Role: LEARNER
I20260808 04:58:47.360677 15932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } }
I20260808 04:58:47.360879 15932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:47.360874 15942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.361094 15942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: This master's current role is: LEARNER
I20260808 04:58:47.362766 15962 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260808 04:58:47.408567 15779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:47.408847 15779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:47.408882 15779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:47.410754 15779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:47.410830 15779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:47.410888 15779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:47.413224 15779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab
--principal=oryx/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:35607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=38211
--tserver_master_addrs=127.3.174.190:35583,127.3.174.189:38099,127.3.174.188:37103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:47.413552 15779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:47.413859 15779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:47.419499 15975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.419703 15980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.419762 15974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:47.420040 15779 server_base.cc:1034] running on GCE node
I20260808 04:58:47.420298 15779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:47.420583 15779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:47.421766 15779 hybrid_clock.cc:648] HybridClock initialized: now 1786165127421743 us; error 44 us; skew 500 ppm
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:47.425493 15779 init.cc:377] Logged in from keytab as oryx/127.3.174.129@KRBTEST.COM (short username oryx)
I20260808 04:58:47.425943 15779 webserver.cc:460] Webserver started at http://127.3.174.129:38211/ using document root <none> and password file <none>
I20260808 04:58:47.426148 15779 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:47.426221 15779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:47.427831 15779 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:47.428473 15993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.428645 15779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:47.428735 15779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "4f0b750dd22c45b79e0b58838242cf2c"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "05fc0ac777aee52d49f4425482d1f5db"
server_key_iv: "cf3a249b8004bae7285a49f239137bad"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.429071 15779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:47.447646 15779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:47.449978 15779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:47.450109 15779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:47.450484 15779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:47.450829 15779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:47.450865 15779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.450896 15779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:47.450913 15779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.465063 15779 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:35607
I20260808 04:58:47.465154 16219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:35607 every 8 connection(s)
I20260808 04:58:47.465579 15779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.189@KRBTEST.COM
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.188@KRBTEST.COM
I20260808 04:58:47.475479 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 15779
I20260808 04:58:47.475646 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 14734
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:47.484027 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.466504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:46053 (local address 127.3.174.188:37103)
0808 04:58:47.466608 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.466610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.467122 (+ 512us) server_negotiation.cc:407] Connection header received
0808 04:58:47.468635 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.468636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.468646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.468744 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.469671 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.470656 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.472176 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.472445 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.472500 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.481040 (+ 8540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.481042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.481042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.481052 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.482591 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.482993 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.482995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.482995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.483052 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.483327 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.483328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.483328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.483462 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.483519 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.483828 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.483922 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":56,"threads_started":1}
I20260808 04:58:47.484459 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.466521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.3.174.129:46053)
0808 04:58:47.466861 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.467003 (+ 142us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.468352 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.468800 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.468801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.469116 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.469576 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.469582 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.470735 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.470736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.472091 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.472094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.472510 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.472511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.472632 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.473328 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.473334 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.480922 (+ 7588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.482673 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.482683 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.482686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.482924 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.483107 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.483108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.483109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.483273 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.483548 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.483556 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.483700 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.483793 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.483965 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":27,"thread_start_us":285,"threads_started":1}
I20260808 04:58:47.485121 16221 heartbeater.cc:347] Connected to a master server at 127.3.174.188:37103
I20260808 04:58:47.485227 16221 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:47.485462 16221 heartbeater.cc:511] Master 127.3.174.188:37103 requested a full tablet report, sending...
I20260808 04:58:47.488962 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.466510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:44391 (local address 127.3.174.190:35583)
0808 04:58:47.466564 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.466566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.467131 (+ 565us) server_negotiation.cc:407] Connection header received
0808 04:58:47.468535 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.468536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.468549 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.468585 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.470546 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.471063 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.472218 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.472554 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.472628 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.486154 (+ 13526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.486156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.486156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.486168 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.488003 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.488320 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.488321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.488322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.488373 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.488511 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.488512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.488512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.488616 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.488648 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.488817 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.488887 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 04:58:47.489256 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.466521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.129:44391)
0808 04:58:47.467035 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.467044 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.468342 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.468621 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.468627 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.469117 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.469605 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.469607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.471119 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.471121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.472156 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.472158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.472636 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.472637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.472759 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.474139 (+ 1380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.474146 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.486039 (+ 11893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.488084 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.488085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.488085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.488254 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.488423 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.488424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.488424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.488466 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.488680 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.488680 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.488725 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.489019 (+ 294us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.489108 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":22,"spinlock_wait_cycles":8448,"thread_start_us":206,"threads_started":1}
I20260808 04:58:47.489774 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.467044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:41899 (local address 127.3.174.189:38099)
0808 04:58:47.467185 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.467187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.467469 (+ 282us) server_negotiation.cc:407] Connection header received
0808 04:58:47.468455 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.468457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.468474 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.468527 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.469703 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.470450 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.472034 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.472265 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.472328 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.486451 (+ 14123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.486452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.486453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.486461 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.488116 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.488569 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.488571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.488571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.488642 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.488906 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.488907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.488908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.489069 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.489121 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.489558 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.489658 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260808 04:58:47.490016 16223 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35583
I20260808 04:58:47.490060 16223 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:47.490123 16223 heartbeater.cc:511] Master 127.3.174.190:35583 requested a full tablet report, sending...
I20260808 04:58:47.490437 14364 ts_manager.cc:194] Re-registered known tserver with Master: 4f0b750dd22c45b79e0b58838242cf2c (127.3.174.129:35607)
I20260808 04:58:47.490510 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.466982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.3.174.129:41899)
0808 04:58:47.467390 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.467399 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.468351 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.468668 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.468669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.469115 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.469578 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.469587 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.470642 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.470643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.471950 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.471953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.472340 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.472341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.472536 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.473158 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.473190 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.486360 (+ 13170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.488218 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.488219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.488220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.488466 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.488743 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.488744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.488744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.488818 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.489237 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.489238 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.489312 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.489947 (+ 635us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.490077 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":95,"thread_start_us":124,"threads_started":1}
I20260808 04:58:47.490871 14364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:44391
I20260808 04:58:47.494908 15870 ts_manager.cc:194] Registered new tserver with Master: 4f0b750dd22c45b79e0b58838242cf2c (127.3.174.129:35607)
I20260808 04:58:47.494948 16227 heartbeater.cc:347] Connected to a master server at 127.3.174.189:38099
I20260808 04:58:47.494995 16227 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:47.495088 16227 heartbeater.cc:511] Master 127.3.174.189:38099 requested a full tablet report, sending...
I20260808 04:58:47.495498 15509 ts_manager.cc:194] Registered new tserver with Master: 4f0b750dd22c45b79e0b58838242cf2c (127.3.174.129:35607)
I20260808 04:58:47.499012 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:36689
--local_ip_for_outbound_sockets=127.3.174.130
--tserver_master_addrs=127.3.174.190:35583
--webserver_port=35347
--webserver_interface=127.3.174.130
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab
--principal=oryx/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.174.190:35583,127.3.174.189:38099
--tserver_master_addrs=127.3.174.190:35583,127.3.174.189:38099,127.3.174.188:37103 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:47.587280 16240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:47.587502 16240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:47.587535 16240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:47.588704 16240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:47.588755 16240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:47.588792 16240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:47.590356 16240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab
--principal=oryx/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:36689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=35347
--tserver_master_addrs=127.3.174.190:35583,127.3.174.189:38099,127.3.174.188:37103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:47.590620 16240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:47.590850 16240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:47.596107 16245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.596150 16248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.596119 16250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:47.596320 16240 server_base.cc:1034] running on GCE node
I20260808 04:58:47.596580 16240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:47.596827 16240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:47.597993 16240 hybrid_clock.cc:648] HybridClock initialized: now 1786165127597980 us; error 27 us; skew 500 ppm
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:47.600664 16240 init.cc:377] Logged in from keytab as oryx/127.3.174.130@KRBTEST.COM (short username oryx)
I20260808 04:58:47.601128 16240 webserver.cc:460] Webserver started at http://127.3.174.130:35347/ using document root <none> and password file <none>
I20260808 04:58:47.601397 16240 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:47.601475 16240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:47.602953 16240 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:47.603611 16261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.603817 16240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:47.603912 16240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "b23c85f2fa75450483b891952ae06569"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "894ab01c556273932f28683a85d5446e"
server_key_iv: "f81f83c93bae1584379f3b125de267f7"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.604291 16240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:47.621797 16240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:47.623677 16240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:47.623800 16240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:47.624035 16240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:47.624366 16240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:47.624409 16240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.624431 16240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:47.624444 16240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.631649 16240 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:36689
I20260808 04:58:47.631707 16461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:36689 every 8 connection(s)
I20260808 04:58:47.632069 16240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:58:47.634603 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 16240
I20260808 04:58:47.634748 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 14991
I20260808 04:58:47.651366 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:43685
--local_ip_for_outbound_sockets=127.3.174.131
--tserver_master_addrs=127.3.174.190:35583
--webserver_port=35851
--webserver_interface=127.3.174.131
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab
--principal=oryx/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.174.190:35583,127.3.174.189:38099
--tserver_master_addrs=127.3.174.190:35583,127.3.174.189:38099,127.3.174.188:37103 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.188@KRBTEST.COM
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.189@KRBTEST.COM
I20260808 04:58:47.655709 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.632790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.3.174.130:51835)
0808 04:58:47.633066 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.633114 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.634191 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.634461 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.634466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.646127 (+ 11661us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.646549 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.646553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.647499 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.647500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.648968 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.648971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.649617 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.649618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.649731 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.650403 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.650415 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.652307 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.654048 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.654055 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.654058 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.654382 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.654595 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.654596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.654596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.654723 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.655008 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.655016 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.655176 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.655298 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.655463 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":61,"thread_start_us":206,"threads_started":1}
I20260808 04:58:47.655764 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.632778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:42555 (local address 127.3.174.190:35583)
0808 04:58:47.632825 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.632827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.633201 (+ 374us) server_negotiation.cc:407] Connection header received
0808 04:58:47.634501 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.634502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.634515 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.634563 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.646664 (+ 12101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.647441 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.649079 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.649428 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.649463 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.653191 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.653192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.653193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.653201 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.654639 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.654989 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.654990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.654990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.655048 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.655233 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.655234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.655234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.655359 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.655398 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.655582 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.655658 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 04:58:47.656173 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.632923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:51835 (local address 127.3.174.188:37103)
0808 04:58:47.632976 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.632978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.633196 (+ 218us) server_negotiation.cc:407] Connection header received
0808 04:58:47.634317 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.634319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.634332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.634376 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.646672 (+ 12296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.647375 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.649069 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.649495 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.649538 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.652402 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.652403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.652404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.652412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.653962 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.654469 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.654470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.654471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.654529 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.654791 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.654792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.654792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.654928 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.654983 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.656000 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.656080 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 04:58:47.656351 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.632789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.130:42555)
0808 04:58:47.633066 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.633114 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.634197 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.634616 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.634616 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.646181 (+ 11565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.646520 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.646530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.647626 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.647627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.648969 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.648972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.649553 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.649554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.649699 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.650414 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.650419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.653074 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.654724 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.654725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.654725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.654909 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.655112 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.655113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.655113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.655165 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.655444 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.655445 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.655479 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.655902 (+ 423us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.656056 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":106,"thread_start_us":180,"threads_started":1}
I20260808 04:58:47.656543 16463 heartbeater.cc:347] Connected to a master server at 127.3.174.188:37103
I20260808 04:58:47.656656 16463 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:47.656849 16463 heartbeater.cc:511] Master 127.3.174.188:37103 requested a full tablet report, sending...
I20260808 04:58:47.657081 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.633197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.3.174.130:38287)
0808 04:58:47.633334 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.633342 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.634200 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.634462 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.634467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.646139 (+ 11672us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.646520 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.646530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.647579 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.647580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.648969 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.648972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.649551 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.649552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.649699 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.650484 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.650488 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.654192 (+ 3704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.655897 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.655898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.655899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.656168 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.656443 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.656444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.656445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.656504 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.656783 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.656784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.656827 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.656890 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.656988 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":66,"thread_start_us":70,"threads_started":1}
I20260808 04:58:47.657164 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.633258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:38287 (local address 127.3.174.189:38099)
0808 04:58:47.633330 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.633332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.633384 (+ 52us) server_negotiation.cc:407] Connection header received
0808 04:58:47.634380 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.634382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.634393 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.634438 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.646665 (+ 12227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.647444 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.649085 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.649407 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.649452 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.654296 (+ 4844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.654298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.654298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.654306 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.655803 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.656318 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.656319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.656319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.656377 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.656562 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.656563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.656564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.656698 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.656756 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.656996 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.657080 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:58:47.657358 15870 ts_manager.cc:194] Registered new tserver with Master: b23c85f2fa75450483b891952ae06569 (127.3.174.130:36689)
I20260808 04:58:47.657497 16464 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35583
I20260808 04:58:47.657541 16466 heartbeater.cc:347] Connected to a master server at 127.3.174.189:38099
I20260808 04:58:47.657541 16464 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:47.657610 16466 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:47.657648 16464 heartbeater.cc:511] Master 127.3.174.190:35583 requested a full tablet report, sending...
I20260808 04:58:47.657678 16466 heartbeater.cc:511] Master 127.3.174.189:38099 requested a full tablet report, sending...
I20260808 04:58:47.657886 14364 ts_manager.cc:194] Re-registered known tserver with Master: b23c85f2fa75450483b891952ae06569 (127.3.174.130:36689)
I20260808 04:58:47.658090 15509 ts_manager.cc:194] Registered new tserver with Master: b23c85f2fa75450483b891952ae06569 (127.3.174.130:36689)
I20260808 04:58:47.658286 14364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:42555
I20260808 04:58:47.684099 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.678409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.0.0.1:37308)
0808 04:58:47.678491 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.678504 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.678562 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.678698 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.678699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.678842 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.678886 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.678887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.679751 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.679753 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.680280 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.680282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.680707 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.680708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.680744 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.681159 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.681163 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.681518 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.683192 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.683194 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.683194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.683357 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.683556 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.683557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.683557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.683607 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.683857 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.683857 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.683896 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.683949 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.684029 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 04:58:47.684156 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.678485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.3.174.188:37103)
0808 04:58:47.678554 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.678556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.678566 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:47.678626 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.678627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.678640 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.678688 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.678945 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.679673 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.680351 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.680645 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.680712 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.681624 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.681625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.681626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.681633 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.683111 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.683428 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.683429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.683429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.683497 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.683660 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.683661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.683661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.683791 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.683857 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.683987 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.684068 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 04:58:47.684406 15887 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 04:58:47.684564 15942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8917e6eeb4234234b3b255985ff749e9. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.684698 15942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: This master's current role is: LEARNER
W20260808 04:58:47.743278 16477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:47.743489 16477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:47.743511 16477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:47.744644 16477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:47.744688 16477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:47.744711 16477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:47.746208 16477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab
--principal=oryx/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:43685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=35851
--tserver_master_addrs=127.3.174.190:35583,127.3.174.189:38099,127.3.174.188:37103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:47.746443 16477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:47.746658 16477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:47.751789 16492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.751806 16494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:47.751819 16499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:47.751984 16477 server_base.cc:1034] running on GCE node
I20260808 04:58:47.752190 16477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:47.752404 16477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:47.753535 16477 hybrid_clock.cc:648] HybridClock initialized: now 1786165127753523 us; error 25 us; skew 500 ppm
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:47.760583 16477 init.cc:377] Logged in from keytab as oryx/127.3.174.131@KRBTEST.COM (short username oryx)
I20260808 04:58:47.761026 16477 webserver.cc:460] Webserver started at http://127.3.174.131:35851/ using document root <none> and password file <none>
I20260808 04:58:47.761250 16477 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:47.761343 16477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:47.762792 16477 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:47.763377 16513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.763531 16477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:47.763620 16477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "6cbd6fbf05d14b67bb4c7601e063004e"
format_stamp: "Formatted at 2026-08-08 04:58:46 on dist-test-slave-gjw7"
server_key: "27ba1324fcb3db947af781584964c43e"
server_key_iv: "5b0cb952d6cad444b2d1c6c44e3c1eb5"
server_key_version: "encryptionkey@0"
I20260808 04:58:47.764003 16477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:47.775625 16477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:47.777756 16477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:47.777886 16477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:47.778123 16477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:47.778417 16477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:47.778450 16477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.778478 16477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:47.778502 16477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:47.787284 16477 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:43685
I20260808 04:58:47.787320 16689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:43685 every 8 connection(s)
I20260808 04:58:47.787691 16477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.189@KRBTEST.COM
Aug 08 04:58:47 dist-test-slave-gjw7 krb5kdc[14295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165127, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.188@KRBTEST.COM
I20260808 04:58:47.797477 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 16477
I20260808 04:58:47.799290 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.789033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:38731 (local address 127.3.174.189:38099)
0808 04:58:47.789081 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.789083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.789180 (+ 97us) server_negotiation.cc:407] Connection header received
0808 04:58:47.790330 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.790331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.790343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.790374 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.791127 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.791853 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.793339 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.793638 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.793677 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.796158 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.796160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.796160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.796168 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.797862 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.798200 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.798201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.798201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.798251 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.798441 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.798442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.798442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.798558 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.798596 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.799118 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.799196 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 04:58:47.799314 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.788654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:43925 (local address 127.3.174.190:35583)
0808 04:58:47.788768 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.788770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.789117 (+ 347us) server_negotiation.cc:407] Connection header received
0808 04:58:47.790383 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.790385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.790397 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.790446 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.791954 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.792504 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.793282 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.793508 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.793563 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.795765 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.795767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.795767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.795776 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.797539 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.798006 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.798007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.798008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.798062 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.798345 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.798346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.798346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.798476 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.798516 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.799120 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.799196 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 04:58:47.799641 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.788751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:57915 (local address 127.3.174.188:37103)
0808 04:58:47.788812 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.788814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.789009 (+ 195us) server_negotiation.cc:407] Connection header received
0808 04:58:47.790309 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.790310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.790324 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.790381 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.791100 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.791857 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.793336 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.793638 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.793677 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.796698 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.796700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.796701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.796709 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.798293 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.798783 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.798784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.798784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.798858 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.799080 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.799081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.799081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.799214 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.799259 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.799471 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.799552 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 04:58:47.799855 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.788931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.3.174.131:38731)
0808 04:58:47.789118 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.789128 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.790231 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.790471 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.790471 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.790850 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.791009 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.791016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.791947 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.791947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.793208 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.793211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.793764 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.793765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.793843 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.794485 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.794489 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.796028 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.797942 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.797943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.797943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.798124 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.798320 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.798321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.798321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.798374 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.798639 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.798646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.798930 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.799414 (+ 484us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.799598 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":121,"thread_start_us":77,"threads_started":1}
I20260808 04:58:47.799867 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.788444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.131:43925)
0808 04:58:47.789061 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.789070 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.790201 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.790458 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.790462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.790846 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.791009 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.791016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.792576 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.792577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.793203 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.793205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.793577 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.793578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.793709 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.794187 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.794198 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.795663 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.797645 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.797653 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.797655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.797927 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.798131 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.798131 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.798132 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.798270 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.798809 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.798810 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.798850 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.799546 (+ 696us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.799642 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":55,"thread_start_us":273,"threads_started":1}
I20260808 04:58:47.800372 16693 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35583
I20260808 04:58:47.800479 16693 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:47.800590 16694 heartbeater.cc:347] Connected to a master server at 127.3.174.189:38099
I20260808 04:58:47.800642 16694 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:47.800683 16693 heartbeater.cc:511] Master 127.3.174.190:35583 requested a full tablet report, sending...
I20260808 04:58:47.800706 16694 heartbeater.cc:511] Master 127.3.174.189:38099 requested a full tablet report, sending...
I20260808 04:58:47.801075 14364 ts_manager.cc:194] Re-registered known tserver with Master: 6cbd6fbf05d14b67bb4c7601e063004e (127.3.174.131:43685)
I20260808 04:58:47.801136 15509 ts_manager.cc:194] Registered new tserver with Master: 6cbd6fbf05d14b67bb4c7601e063004e (127.3.174.131:43685)
I20260808 04:58:47.801470 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.788444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.3.174.131:57915)
0808 04:58:47.788880 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.788921 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.790193 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.790457 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.790461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.790787 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:47.791008 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.791013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.791942 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.791942 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.793208 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.793211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.793765 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.793765 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.793844 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.794404 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:47.794408 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:47.796574 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:47.798542 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.798543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.798543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.798721 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.798938 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:47.798939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:47.798939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:47.799005 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:47.799314 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:47.799315 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:47.799358 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.799777 (+ 419us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.799880 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":14,"thread_start_us":291,"threads_started":1}
I20260808 04:58:47.801601 14364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:43925
I20260808 04:58:47.801764 16690 heartbeater.cc:347] Connected to a master server at 127.3.174.188:37103
I20260808 04:58:47.801818 16690 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:47.801882 16690 heartbeater.cc:511] Master 127.3.174.188:37103 requested a full tablet report, sending...
I20260808 04:58:47.802153 15870 ts_manager.cc:194] Registered new tserver with Master: 6cbd6fbf05d14b67bb4c7601e063004e (127.3.174.131:43685)
I20260808 04:58:47.808207 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.799691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.3.174.188:37103)
0808 04:58:47.799764 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.799765 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.801256 (+ 1491us) server_negotiation.cc:407] Connection header received
0808 04:58:47.801394 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.801395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.801404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.801429 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.802649 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.803171 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.804113 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.804371 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.804404 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.805680 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.805681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.805681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.805688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.807172 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.807511 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.807512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.807512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.807563 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.807725 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.807726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.807726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.807859 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.807944 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.808064 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.808127 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":60}
I20260808 04:58:47.808204 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.799640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.3.174.189:38099)
0808 04:58:47.799685 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.799687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.800122 (+ 435us) server_negotiation.cc:407] Connection header received
0808 04:58:47.800315 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.800316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.800325 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.800349 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.801180 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.801878 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.802865 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.803155 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.803223 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.805606 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.805607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.805607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.805613 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.807010 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.807334 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.807335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.807335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.807388 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.807609 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.807610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.807610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.807731 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.807806 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.808042 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.808116 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 04:58:47.809074 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.800617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.3.174.190:35583)
0808 04:58:47.800718 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.800720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.802496 (+ 1776us) server_negotiation.cc:407] Connection header received
0808 04:58:47.802525 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.802525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.802535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.802591 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.803234 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.803945 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.804902 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.805093 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.805146 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.806340 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.806341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.806342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.806348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.807938 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.808361 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.808362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.808363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.808425 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.808599 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.808600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.808601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.808739 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.808810 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.808943 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.809010 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 04:58:47.810365 14364 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:47.812268 14364 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:47.812939 15780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: NON_VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: true } } }
I20260808 04:58:47.813026 15780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:47.813894 15780 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: attempting to promote NON_VOTER 36bd4278aaa94063923d5e6ffddef9e4 to VOTER
I20260808 04:58:47.814013 15780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } }
I20260808 04:58:47.814591 15534 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 FOLLOWER]: Refusing update from remote peer 8917e6eeb4234234b3b255985ff749e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 04:58:47.814630 15887 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 LEARNER]: Refusing update from remote peer 8917e6eeb4234234b3b255985ff749e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 04:58:47.814824 16713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 04:58:47.814931 16714 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 04:58:47.815582 16713 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 36bd4278aaa94063923d5e6ffddef9e4 (127.3.174.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:47.815733 15887 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 36bd4278aaa94063923d5e6ffddef9e4 (127.3.174.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:47.816591 15942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:47.816617 16713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:47.816675 15942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:58:47.816674 16713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:47.816748 16713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:47.816746 15780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:47.816823 16713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:47.816829 15780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:47.816947 15534 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 36bd4278aaa94063923d5e6ffddef9e4 (127.3.174.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:47.818387 16729 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8917e6eeb4234234b3b255985ff749e9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:47.818476 16729 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:58:47.821625 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.817921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:36689 (local address 127.0.0.1:52560)
0808 04:58:47.818250 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.818259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.818304 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.818829 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.818830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.818831 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:47.818869 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.818870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.819823 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.819824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.820693 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.820695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.821397 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.821399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.821470 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.821476 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.821506 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.821543 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":40,"thread_start_us":124,"threads_started":1}
I20260808 04:58:47.822069 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.818223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.3.174.130:36689)
0808 04:58:47.818601 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.818605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.818614 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:47.818707 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.818710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.818726 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.818935 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:47.818990 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.819711 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.820769 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.821257 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.821736 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.821868 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.821897 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.821944 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":84,"threads_started":1}
I20260808 04:58:47.822530 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.817876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:35607 (local address 127.0.0.1:58390)
0808 04:58:47.818322 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.818334 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.818397 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.818765 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.818766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.818768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:47.818818 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.818820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.819782 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.819783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.821339 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.821342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.822313 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.822314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.822377 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.822381 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.822423 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.822461 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":25}
I20260808 04:58:47.822661 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.818218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.3.174.129:35607)
0808 04:58:47.818601 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.818606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.818617 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:58:47.818661 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.818663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.818677 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.818763 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:47.818876 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.819695 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.821397 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.822220 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.822290 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.822460 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.822488 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.822533 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":240,"threads_started":1}
I20260808 04:58:47.823149 16343 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdefe99bd9841afb6068c6daa8eb374 (DEFAULT_TABLE table=test-table [id=7a5fcefc60424df0b765f51ff7ed2a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:47.823464 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.817876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:43685 (local address 127.0.0.1:53014)
0808 04:58:47.818069 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.818081 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.818123 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.820061 (+ 1938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.820062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.820064 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:47.820098 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.820099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.821406 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.821407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.822381 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.822384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.823256 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.823257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.823317 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.823321 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.823365 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.823401 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":19,"thread_start_us":86,"threads_started":1}
I20260808 04:58:47.823609 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.818058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53014 (local address 127.3.174.131:43685)
0808 04:58:47.819897 (+ 1839us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.819901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.819910 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:47.819949 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.819951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.819968 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.820044 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:47.820155 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.821248 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.822459 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.823169 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.823243 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.823409 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.823440 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.823479 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1798,"thread_start_us":163,"threads_started":1}
I20260808 04:58:47.823498 16093 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdefe99bd9841afb6068c6daa8eb374 (DEFAULT_TABLE table=test-table [id=7a5fcefc60424df0b765f51ff7ed2a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:47.823616 16343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdefe99bd9841afb6068c6daa8eb374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:47.823849 16093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdefe99bd9841afb6068c6daa8eb374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:47.826207 16590 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdefe99bd9841afb6068c6daa8eb374 (DEFAULT_TABLE table=test-table [id=7a5fcefc60424df0b765f51ff7ed2a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:47.826552 16590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bdefe99bd9841afb6068c6daa8eb374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:47.828061 16735 tablet_bootstrap.cc:492] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: Bootstrap starting.
I20260808 04:58:47.828478 16739 tablet_bootstrap.cc:492] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: Bootstrap starting.
I20260808 04:58:47.829361 16735 tablet_bootstrap.cc:654] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:47.829571 16739 tablet_bootstrap.cc:654] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:47.829981 16735 log.cc:824] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:47.830448 16739 log.cc:824] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:47.830904 16735 tablet_bootstrap.cc:492] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: No bootstrap required, opened a new log
I20260808 04:58:47.830983 16735 ts_tablet_manager.cc:1397] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:58:47.831346 16739 tablet_bootstrap.cc:492] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: No bootstrap required, opened a new log
I20260808 04:58:47.831420 16739 ts_tablet_manager.cc:1397] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 04:58:47.831624 16744 tablet_bootstrap.cc:492] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: Bootstrap starting.
I20260808 04:58:47.832641 16744 tablet_bootstrap.cc:654] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:47.833178 16744 log.cc:824] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:47.833935 16744 tablet_bootstrap.cc:492] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: No bootstrap required, opened a new log
I20260808 04:58:47.833995 16744 ts_tablet_manager.cc:1397] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:58:47.835487 16739 raft_consensus.cc:348] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } }
I20260808 04:58:47.835670 16739 raft_consensus.cc:374] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:47.835721 16739 raft_consensus.cc:729] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f0b750dd22c45b79e0b58838242cf2c, State: Initialized, Role: FOLLOWER
I20260808 04:58:47.835858 16739 consensus_queue.cc:260] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } }
I20260808 04:58:47.835902 16735 raft_consensus.cc:348] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } }
I20260808 04:58:47.836066 16735 raft_consensus.cc:374] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:47.836115 16735 raft_consensus.cc:729] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b23c85f2fa75450483b891952ae06569, State: Initialized, Role: FOLLOWER
I20260808 04:58:47.836261 16739 ts_tablet_manager.cc:1428] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260808 04:58:47.836248 16735 consensus_queue.cc:260] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } }
I20260808 04:58:47.836356 16223 heartbeater.cc:503] Master 127.3.174.190:35583 was elected leader, sending a full tablet report...
I20260808 04:58:47.836566 16735 ts_tablet_manager.cc:1428] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260808 04:58:47.836992 16744 raft_consensus.cc:348] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } }
I20260808 04:58:47.837133 16744 raft_consensus.cc:374] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:47.837168 16744 raft_consensus.cc:729] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cbd6fbf05d14b67bb4c7601e063004e, State: Initialized, Role: FOLLOWER
I20260808 04:58:47.837313 16464 heartbeater.cc:503] Master 127.3.174.190:35583 was elected leader, sending a full tablet report...
I20260808 04:58:47.837333 16744 consensus_queue.cc:260] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } }
I20260808 04:58:47.837601 16744 ts_tablet_manager.cc:1428] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 04:58:47.837716 16693 heartbeater.cc:503] Master 127.3.174.190:35583 was elected leader, sending a full tablet report...
W20260808 04:58:47.883198 16469 tablet.cc:2369] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:47.935467 16748 raft_consensus.cc:482] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:47.935590 16748 raft_consensus.cc:504] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } }
I20260808 04:58:47.935930 16748 leader_election.cc:288] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cbd6fbf05d14b67bb4c7601e063004e (127.3.174.131:43685), b23c85f2fa75450483b891952ae06569 (127.3.174.130:36689)
I20260808 04:58:47.939003 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.936173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:43685 (local address 127.3.174.129:42713)
0808 04:58:47.936220 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.936231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.936269 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.936506 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.936507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.936508 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:47.936724 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.936726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.937343 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.937344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.938196 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.938199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.938695 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.938696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.938837 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.938840 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.938888 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.938926 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 04:58:47.939003 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.936341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:42713 (local address 127.3.174.131:43685)
0808 04:58:47.936395 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.936397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.936404 (+ 7us) server_negotiation.cc:407] Connection header received
0808 04:58:47.936427 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.936428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.936436 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.936483 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:47.936798 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.937244 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.938290 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.938628 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.938689 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.938699 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.938903 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.938942 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 04:58:47.939105 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.936257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:57291 (local address 127.3.174.130:36689)
0808 04:58:47.936319 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.936321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.936329 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:47.936355 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.936355 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.936365 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.936421 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:47.936644 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.937364 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.938307 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.938674 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.938745 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.938755 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.939002 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.939047 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 04:58:47.939177 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.936107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:36689 (local address 127.3.174.129:57291)
0808 04:58:47.936173 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:47.936186 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:47.936236 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:47.936433 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:47.936434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:47.936435 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:47.936571 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.936573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.937435 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.937436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.938209 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:47.938211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.938748 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.938748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:47.938932 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.938935 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:47.938984 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 04:58:47.939024 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 04:58:47.939561 16380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdefe99bd9841afb6068c6daa8eb374" candidate_uuid: "4f0b750dd22c45b79e0b58838242cf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b23c85f2fa75450483b891952ae06569" is_pre_election: true
I20260808 04:58:47.939679 16616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdefe99bd9841afb6068c6daa8eb374" candidate_uuid: "4f0b750dd22c45b79e0b58838242cf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" is_pre_election: true
I20260808 04:58:47.939734 16380 raft_consensus.cc:2454] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f0b750dd22c45b79e0b58838242cf2c in term 0.
I20260808 04:58:47.939831 16616 raft_consensus.cc:2454] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f0b750dd22c45b79e0b58838242cf2c in term 0.
I20260808 04:58:47.939951 16002 leader_election.cc:302] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f0b750dd22c45b79e0b58838242cf2c, b23c85f2fa75450483b891952ae06569; no voters:
I20260808 04:58:47.940073 16748 raft_consensus.cc:2781] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:47.940136 16748 raft_consensus.cc:482] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:47.940152 16748 raft_consensus.cc:3037] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:47.941406 16748 raft_consensus.cc:504] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } }
I20260808 04:58:47.941529 16748 leader_election.cc:288] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [CANDIDATE]: Term 1 election: Requested vote from peers 6cbd6fbf05d14b67bb4c7601e063004e (127.3.174.131:43685), b23c85f2fa75450483b891952ae06569 (127.3.174.130:36689)
I20260808 04:58:47.941699 16616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdefe99bd9841afb6068c6daa8eb374" candidate_uuid: "4f0b750dd22c45b79e0b58838242cf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cbd6fbf05d14b67bb4c7601e063004e"
I20260808 04:58:47.941721 16380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdefe99bd9841afb6068c6daa8eb374" candidate_uuid: "4f0b750dd22c45b79e0b58838242cf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b23c85f2fa75450483b891952ae06569"
I20260808 04:58:47.941781 16616 raft_consensus.cc:3037] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:47.941792 16380 raft_consensus.cc:3037] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:47.942564 16616 raft_consensus.cc:2454] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f0b750dd22c45b79e0b58838242cf2c in term 1.
I20260808 04:58:47.942792 16003 leader_election.cc:302] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f0b750dd22c45b79e0b58838242cf2c, 6cbd6fbf05d14b67bb4c7601e063004e; no voters:
I20260808 04:58:47.942879 16748 raft_consensus.cc:2781] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:47.942996 16748 raft_consensus.cc:686] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 1 LEADER]: Becoming Leader. State: Replica: 4f0b750dd22c45b79e0b58838242cf2c, State: Running, Role: LEADER
I20260808 04:58:47.943087 16748 consensus_queue.cc:237] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } }
I20260808 04:58:47.943166 16380 raft_consensus.cc:2454] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f0b750dd22c45b79e0b58838242cf2c in term 1.
I20260808 04:58:47.943971 14365 catalog_manager.cc:5495] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f0b750dd22c45b79e0b58838242cf2c (127.3.174.129). New cstate: current_term: 1 leader_uuid: "4f0b750dd22c45b79e0b58838242cf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } health_report { overall_health: UNKNOWN } } }
I20260808 04:58:47.965595 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.963277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.3.174.129:35607)
0808 04:58:47.963329 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.963331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.963340 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:47.963410 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.963411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.963429 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.963479 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:47.963660 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.964159 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.964892 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.965073 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.965127 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.965267 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:47.965404 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:47.965479 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.965499 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.965521 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 04:58:47.967267 16232 tablet.cc:2369] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:47.967970 16380 raft_consensus.cc:1261] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [term 1 FOLLOWER]: Refusing update from remote peer 4f0b750dd22c45b79e0b58838242cf2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:47.967948 16616 raft_consensus.cc:1261] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [term 1 FOLLOWER]: Refusing update from remote peer 4f0b750dd22c45b79e0b58838242cf2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:47.968367 16748 consensus_queue.cc:1035] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:47.968567 16754 consensus_queue.cc:1035] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:47.972242 16758 mvcc.cc:205] Tried to move back new op lower bound from 7316132364153348096 to 7316132364055584768. Current Snapshot: MvccSnapshot[applied={T|T < 7316132364153348096}]
I20260808 04:58:47.972481 16759 mvcc.cc:205] Tried to move back new op lower bound from 7316132364153348096 to 7316132364055584768. Current Snapshot: MvccSnapshot[applied={T|T < 7316132364153348096}]
I20260808 04:58:47.973417 16766 mvcc.cc:205] Tried to move back new op lower bound from 7316132364153348096 to 7316132364055584768. Current Snapshot: MvccSnapshot[applied={T|T < 7316132364153348096}]
I20260808 04:58:47.976476 14365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:47.976562 14365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:47.978080 14365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8917e6eeb4234234b3b255985ff749e9: Sending DeleteTablet for 3 replicas of tablet 2bdefe99bd9841afb6068c6daa8eb374
I20260808 04:58:47.978531 16093 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bdefe99bd9841afb6068c6daa8eb374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:47 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:58390
I20260808 04:58:47.978554 16590 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bdefe99bd9841afb6068c6daa8eb374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:47 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:53014
I20260808 04:58:47.978955 16781 tablet_replica.cc:330] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: stopping tablet replica
I20260808 04:58:47.979059 16781 raft_consensus.cc:2229] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:47.979167 16781 raft_consensus.cc:2258] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:47.979223 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 14315
I20260808 04:58:47.979226 16343 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bdefe99bd9841afb6068c6daa8eb374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:47 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:52560
I20260808 04:58:47.989313 16781 ts_tablet_manager.cc:1905] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:47.989531 16786 tablet_replica.cc:330] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: stopping tablet replica
I20260808 04:58:47.989663 16786 raft_consensus.cc:2229] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:47.989784 16786 raft_consensus.cc:2258] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:47.990312 16786 ts_tablet_manager.cc:1905] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:47.991107 16781 ts_tablet_manager.cc:1918] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:47.991192 16781 log.cc:1196] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/2bdefe99bd9841afb6068c6daa8eb374
I20260808 04:58:47.991500 16781 ts_tablet_manager.cc:1939] T 2bdefe99bd9841afb6068c6daa8eb374 P 6cbd6fbf05d14b67bb4c7601e063004e: Deleting consensus metadata
I20260808 04:58:47.992074 16786 ts_tablet_manager.cc:1918] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:47.992141 16786 log.cc:1196] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/2bdefe99bd9841afb6068c6daa8eb374
I20260808 04:58:47.992358 16786 ts_tablet_manager.cc:1939] T 2bdefe99bd9841afb6068c6daa8eb374 P b23c85f2fa75450483b891952ae06569: Deleting consensus metadata
I20260808 04:58:47.992913 16787 tablet_replica.cc:330] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: stopping tablet replica
I20260808 04:58:47.993031 16787 raft_consensus.cc:2229] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:47.993175 16787 raft_consensus.cc:2258] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:47.993781 16787 ts_tablet_manager.cc:1905] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:47.995247 16787 ts_tablet_manager.cc:1918] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:47.995324 16787 log.cc:1196] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/2bdefe99bd9841afb6068c6daa8eb374
I20260808 04:58:47.995579 16787 ts_tablet_manager.cc:1939] T 2bdefe99bd9841afb6068c6daa8eb374 P 4f0b750dd22c45b79e0b58838242cf2c: Deleting consensus metadata
I20260808 04:58:47.997365 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.988538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43454 (local address 127.3.174.189:38099)
0808 04:58:47.988609 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.988611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.990015 (+ 1404us) server_negotiation.cc:407] Connection header received
0808 04:58:47.990126 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.990127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.990139 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.990168 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.990940 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.991669 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.992667 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.992910 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.992980 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:47.994500 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:47.994502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:47.994502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:47.994510 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:47.996159 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.996562 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.996563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.996563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.996623 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:47.996819 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:47.996819 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:47.996820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:47.996926 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:47.996995 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:47.997123 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 04:58:47.997238 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 04:58:48.000196 16267 connection.cc:535] server connection from 127.0.0.1:52560 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 04:58:48.000214 15998 connection.cc:535] server connection from 127.0.0.1:58390 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 04:58:48.000221 16517 connection.cc:535] server connection from 127.0.0.1:53014 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 04:58:48.003186 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:47.979050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37322 (local address 127.3.174.188:37103)
0808 04:58:47.979097 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:47.979099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:47.997403 (+ 18304us) server_negotiation.cc:407] Connection header received
0808 04:58:47.997504 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:47.997505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:47.997518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:47.997545 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:47.998139 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.998716 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.999566 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:47.999754 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:47.999806 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:48.000939 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:48.000940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:48.000941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:48.000948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:48.002236 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:48.002525 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:48.002526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:48.002526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:48.002579 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:48.002739 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:48.002740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:48.002740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:48.002846 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:48.002927 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:48.003030 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 04:58:48.003098 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:58:48.014112 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:48.007002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43466 (local address 127.3.174.189:38099)
0808 04:58:48.007056 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:48.007058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:48.007206 (+ 148us) server_negotiation.cc:407] Connection header received
0808 04:58:48.007312 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:48.007313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:48.007323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:48.007350 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:48.008384 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:48.009010 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:48.009799 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:48.010085 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:48.010148 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:48.011559 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:48.011560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:48.011560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:48.011567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:48.013044 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:48.013392 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:48.013393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:48.013393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:48.013445 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:48.013613 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:48.013614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:48.013614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:48.013741 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:48.013837 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:48.013942 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 04:58:48.014017 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 04:58:48.014535 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:48.007242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.3.174.188:37103)
0808 04:58:48.007281 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:48.007283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:48.007409 (+ 126us) server_negotiation.cc:407] Connection header received
0808 04:58:48.007429 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:48.007430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:48.007439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:48.007485 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:48.008127 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:48.008751 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:48.009863 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:48.010143 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:48.010196 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:48.011477 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:48.011478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:48.011479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:48.011486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:48.013347 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:48.013721 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:48.013722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:48.013722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:48.013783 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:48.013982 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:48.013983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:48.013983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:48.014105 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:48.014172 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:48.014390 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 04:58:48.014471 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 04:58:48.363339 15962 catalog_manager.cc:1259] Loaded cluster ID: 10dd19f8c9d84032a41440f7466a608b
I20260808 04:58:48.363413 15962 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: loading cluster ID for follower catalog manager: success
I20260808 04:58:48.364156 15962 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: acquiring CA information for follower catalog manager: success
I20260808 04:58:48.364395 15962 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 04:58:48.973672 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:48.973397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.131:47179)
0808 04:58:48.973526 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:48.973614 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
W20260808 04:58:48.973906 16693 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:35583 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
I20260808 04:58:48.974385 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:48.974132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.130:57587)
0808 04:58:48.974271 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:48.974340 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
W20260808 04:58:48.974565 16464 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:35583 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
I20260808 04:58:48.974601 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:48.974394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.129:46937)
0808 04:58:48.974508 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:48.974565 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":50,"threads_started":1}
W20260808 04:58:48.974779 16223 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:35583 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
I20260808 04:58:49.478442 16805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8917e6eeb4234234b3b255985ff749e9)
I20260808 04:58:49.478555 16805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } }
I20260808 04:58:49.478767 16805 leader_election.cc:288] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8917e6eeb4234234b3b255985ff749e9 (127.3.174.190:35583), ff9ef9171b664a6980dc877479e811be (127.3.174.189:38099)
I20260808 04:58:49.479025 16806 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8917e6eeb4234234b3b255985ff749e9)
I20260808 04:58:49.479140 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.478904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56846)
0808 04:58:49.479017 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.479088 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":47,"threads_started":1}
I20260808 04:58:49.479172 16806 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } }
W20260808 04:58:49.479326 15847 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
I20260808 04:58:49.479627 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.479447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56852)
0808 04:58:49.479538 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.479589 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":41,"threads_started":1}
I20260808 04:58:49.479737 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.479625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56858)
0808 04:58:49.479664 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.479689 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 04:58:49.479835 15847 leader_election.cc:334] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8917e6eeb4234234b3b255985ff749e9 (127.3.174.190:35583): Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
W20260808 04:58:49.479910 15477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
I20260808 04:58:49.480108 16806 leader_election.cc:288] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8917e6eeb4234234b3b255985ff749e9 (127.3.174.190:35583), 36bd4278aaa94063923d5e6ffddef9e4 (127.3.174.188:37103)
I20260808 04:58:49.480482 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.480282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56866)
0808 04:58:49.480412 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.480445 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260808 04:58:49.480545 15477 leader_election.cc:334] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8917e6eeb4234234b3b255985ff749e9 (127.3.174.190:35583): Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
I20260808 04:58:49.482681 16810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.478989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43470 (local address 127.3.174.189:38099)
0808 04:58:49.479097 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:49.479099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:49.479171 (+ 72us) server_negotiation.cc:407] Connection header received
0808 04:58:49.479326 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:49.479327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:49.479338 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:49.479391 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:49.479606 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.480342 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.481703 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.482259 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.482328 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.482342 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:49.482538 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 04:58:49.482584 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":55,"threads_started":1}
I20260808 04:58:49.482712 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.478944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:38099 (local address 127.0.0.1:43470)
0808 04:58:49.479092 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.479104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:49.479194 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:49.479443 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:49.479444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:49.479446 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:49.479508 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.479510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.480442 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.480444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.481617 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.481621 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.482350 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.482351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.482415 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.482442 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 04:58:49.482530 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 04:58:49.482635 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":57,"thread_start_us":78,"threads_started":1}
I20260808 04:58:49.483103 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36bd4278aaa94063923d5e6ffddef9e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff9ef9171b664a6980dc877479e811be" is_pre_election: true
I20260808 04:58:49.483189 15534 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36bd4278aaa94063923d5e6ffddef9e4 in term 1.
I20260808 04:58:49.483319 15843 leader_election.cc:302] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36bd4278aaa94063923d5e6ffddef9e4, ff9ef9171b664a6980dc877479e811be; no voters: 8917e6eeb4234234b3b255985ff749e9
I20260808 04:58:49.483399 16805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 04:58:49.483438 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.479720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:37103 (local address 127.0.0.1:37338)
0808 04:58:49.480019 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.480031 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:49.480092 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:49.480296 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:49.480297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:49.480300 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:49.480356 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.480357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.481232 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.481233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.482381 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.482383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.483208 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.483209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.483275 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.483280 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:49.483330 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 04:58:49.483370 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260808 04:58:49.483448 16805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8917e6eeb4234234b3b255985ff749e9)
I20260808 04:58:49.483481 16805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 1 FOLLOWER]: Advancing to term 2
I20260808 04:58:49.483600 16816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.479845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37338 (local address 127.3.174.188:37103)
0808 04:58:49.479978 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:49.479980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:49.480142 (+ 162us) server_negotiation.cc:407] Connection header received
0808 04:58:49.480167 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:49.480168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:49.480189 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:49.480252 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:49.480428 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.481127 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.482460 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.483033 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.483104 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.483169 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:49.483359 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 04:58:49.483400 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260808 04:58:49.483820 15887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff9ef9171b664a6980dc877479e811be" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "36bd4278aaa94063923d5e6ffddef9e4" is_pre_election: true
I20260808 04:58:49.484344 16805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } }
I20260808 04:58:49.484472 15887 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff9ef9171b664a6980dc877479e811be in current term 2: Already voted for candidate 36bd4278aaa94063923d5e6ffddef9e4 in this term.
I20260808 04:58:49.484642 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.484560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56882)
0808 04:58:49.484593 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.484622 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 04:58:49.484647 15476 leader_election.cc:302] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff9ef9171b664a6980dc877479e811be; no voters: 36bd4278aaa94063923d5e6ffddef9e4, 8917e6eeb4234234b3b255985ff749e9
I20260808 04:58:49.484475 16805 leader_election.cc:288] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [CANDIDATE]: Term 2 election: Requested vote from peers 8917e6eeb4234234b3b255985ff749e9 (127.3.174.190:35583), ff9ef9171b664a6980dc877479e811be (127.3.174.189:38099)
I20260808 04:58:49.484756 16806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 1 FOLLOWER]: Advancing to term 2
I20260808 04:58:49.484931 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.484861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56896)
0808 04:58:49.484896 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.484918 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 04:58:49.484985 15847 leader_election.cc:334] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8917e6eeb4234234b3b255985ff749e9 (127.3.174.190:35583): Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
I20260808 04:58:49.485369 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36bd4278aaa94063923d5e6ffddef9e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff9ef9171b664a6980dc877479e811be"
I20260808 04:58:49.485558 16806 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 04:58:49.486136 15534 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36bd4278aaa94063923d5e6ffddef9e4 in term 2.
I20260808 04:58:49.486306 15843 leader_election.cc:302] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36bd4278aaa94063923d5e6ffddef9e4, ff9ef9171b664a6980dc877479e811be; no voters: 8917e6eeb4234234b3b255985ff749e9
I20260808 04:58:49.486419 16805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 04:58:49.486568 16805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [term 2 LEADER]: Becoming Leader. State: Replica: 36bd4278aaa94063923d5e6ffddef9e4, State: Running, Role: LEADER
I20260808 04:58:49.486640 16805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } }
I20260808 04:58:49.486972 16821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36bd4278aaa94063923d5e6ffddef9e4. Latest consensus state: current_term: 2 leader_uuid: "36bd4278aaa94063923d5e6ffddef9e4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:49.487067 16821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:49.487243 16823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:49.487674 16823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:49.487751 16823 catalog_manager.cc:1259] Loaded cluster ID: 10dd19f8c9d84032a41440f7466a608b
I20260808 04:58:49.487780 16823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:49.487924 16823 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:49.487988 16823 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Loaded TSK: 0
I20260808 04:58:49.488116 16823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:58:49.491297 15865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:49.492841 15865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:49.493703 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.493623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56928)
0808 04:58:49.493657 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.493683 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 04:58:49.493724 15534 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [term 2 FOLLOWER]: Refusing update from remote peer 36bd4278aaa94063923d5e6ffddef9e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 04:58:49.493906 16821 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 04:58:49.494021 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.493930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.0.0.1:56930)
0808 04:58:49.493975 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.493999 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 04:58:49.494086 15847 consensus_peers.cc:487] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 -> Peer 8917e6eeb4234234b3b255985ff749e9 (127.3.174.190:35583): Couldn't send request to peer 8917e6eeb4234234b3b255985ff749e9. Status: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 04:58:49.494540 16806 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36bd4278aaa94063923d5e6ffddef9e4. Latest consensus state: current_term: 2 leader_uuid: "36bd4278aaa94063923d5e6ffddef9e4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:49.494628 16806 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:58:49.495003 16821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "36bd4278aaa94063923d5e6ffddef9e4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:49.495083 16821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:49.495100 16806 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "36bd4278aaa94063923d5e6ffddef9e4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8917e6eeb4234234b3b255985ff749e9" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35583 } } peers { permanent_uuid: "ff9ef9171b664a6980dc877479e811be" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 38099 } attrs { promote: false } } peers { permanent_uuid: "36bd4278aaa94063923d5e6ffddef9e4" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 37103 } attrs { promote: false } } }
I20260808 04:58:49.495182 16806 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff9ef9171b664a6980dc877479e811be [sys.catalog]: This master's current role is: FOLLOWER
I20260808 04:58:49.499567 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.496666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:43685 (local address 127.0.0.1:53026)
0808 04:58:49.496807 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.496818 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:49.496852 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:49.497070 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:49.497071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:49.497073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:49.497116 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.497118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.497820 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.497821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.498745 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.498747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.499355 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.499356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.499419 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.499425 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 04:58:49.499465 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 04:58:49.499503 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":107,"threads_started":1}
I20260808 04:58:49.499579 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.496732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.3.174.131:43685)
0808 04:58:49.496831 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:49.496833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:49.496930 (+ 97us) server_negotiation.cc:407] Connection header received
0808 04:58:49.496959 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:49.496960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:49.496970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:49.497173 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:49.497233 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.497751 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.498827 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.499275 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.499336 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.499349 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:49.499476 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 04:58:49.499516 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":49,"threads_started":1}
I20260808 04:58:49.499987 16590 tablet_service.cc:1467] Processing CreateTablet for tablet b775f4d8b6f5477da1a2daef0a96c27e (DEFAULT_TABLE table=test-table [id=f4709e9b396b4a9cb61f5fae95d2ef26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:49.500098 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.496600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:35607 (local address 127.0.0.1:58406)
0808 04:58:49.496645 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.496654 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:49.496698 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:49.497164 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:49.497165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:49.497168 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:49.497197 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.497198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.498124 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.498125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.499135 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.499137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.499886 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.499888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.499950 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.499953 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:49.500006 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 04:58:49.500043 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":23}
I20260808 04:58:49.500124 16590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b775f4d8b6f5477da1a2daef0a96c27e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:49.500140 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.496700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.3.174.129:35607)
0808 04:58:49.496932 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:49.496934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:49.496941 (+ 7us) server_negotiation.cc:407] Connection header received
0808 04:58:49.496965 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:49.496966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:49.496976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:49.497161 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:49.497274 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.498031 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.499211 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.499781 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.499874 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.499886 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:49.500025 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 04:58:49.500078 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":77,"threads_started":1}
I20260808 04:58:49.500981 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.496606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:36689 (local address 127.0.0.1:52562)
0808 04:58:49.496824 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.496831 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:49.496867 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:49.497091 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:49.497092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:49.497094 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:49.497274 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.497276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.498172 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.498173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.500139 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.500142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.500820 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.500821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.500865 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.500869 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:49.500912 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 04:58:49.500940 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":3}
I20260808 04:58:49.501039 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.496867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.3.174.130:36689)
0808 04:58:49.496949 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:49.496951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:49.496985 (+ 34us) server_negotiation.cc:407] Connection header received
0808 04:58:49.497010 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:49.497011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:49.497019 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:49.497075 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:49.497379 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.498098 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.500242 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.500742 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.500815 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.500826 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:49.500930 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 04:58:49.500972 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62,"thread_start_us":41,"threads_started":1}
I20260808 04:58:49.501257 16343 tablet_service.cc:1467] Processing CreateTablet for tablet b775f4d8b6f5477da1a2daef0a96c27e (DEFAULT_TABLE table=test-table [id=f4709e9b396b4a9cb61f5fae95d2ef26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:49.501426 16343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b775f4d8b6f5477da1a2daef0a96c27e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:49.501581 16093 tablet_service.cc:1467] Processing CreateTablet for tablet b775f4d8b6f5477da1a2daef0a96c27e (DEFAULT_TABLE table=test-table [id=f4709e9b396b4a9cb61f5fae95d2ef26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:49.501689 16093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b775f4d8b6f5477da1a2daef0a96c27e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:49.502578 16848 tablet_bootstrap.cc:492] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: Bootstrap starting.
I20260808 04:58:49.502662 16849 tablet_bootstrap.cc:492] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c: Bootstrap starting.
I20260808 04:58:49.502961 16847 tablet_bootstrap.cc:492] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: Bootstrap starting.
I20260808 04:58:49.502978 16848 tablet_bootstrap.cc:654] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:49.503003 16849 tablet_bootstrap.cc:654] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:49.503311 16847 tablet_bootstrap.cc:654] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:49.503542 16849 tablet_bootstrap.cc:492] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c: No bootstrap required, opened a new log
I20260808 04:58:49.503536 16848 tablet_bootstrap.cc:492] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: No bootstrap required, opened a new log
I20260808 04:58:49.503587 16848 ts_tablet_manager.cc:1397] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:49.503674 16849 ts_tablet_manager.cc:1397] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:49.503701 16848 raft_consensus.cc:348] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } }
I20260808 04:58:49.503772 16848 raft_consensus.cc:374] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:49.503789 16848 raft_consensus.cc:729] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b23c85f2fa75450483b891952ae06569, State: Initialized, Role: FOLLOWER
I20260808 04:58:49.503784 16849 raft_consensus.cc:348] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } }
I20260808 04:58:49.503829 16847 tablet_bootstrap.cc:492] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: No bootstrap required, opened a new log
I20260808 04:58:49.503839 16849 raft_consensus.cc:374] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:49.503860 16849 raft_consensus.cc:729] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f0b750dd22c45b79e0b58838242cf2c, State: Initialized, Role: FOLLOWER
I20260808 04:58:49.503845 16848 consensus_queue.cc:260] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } }
I20260808 04:58:49.503870 16847 ts_tablet_manager.cc:1397] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:49.503896 16849 consensus_queue.cc:260] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } }
I20260808 04:58:49.504009 16849 ts_tablet_manager.cc:1428] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:49.504005 16848 ts_tablet_manager.cc:1428] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:49.504418 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.504211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.130:43773)
0808 04:58:49.504341 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.504379 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":87,"threads_started":1}
I20260808 04:58:49.504011 16847 raft_consensus.cc:348] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } }
I20260808 04:58:49.504812 16847 raft_consensus.cc:374] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:49.504846 16847 raft_consensus.cc:729] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cbd6fbf05d14b67bb4c7601e063004e, State: Initialized, Role: FOLLOWER
I20260808 04:58:49.504889 16860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.504263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.129:51709)
0808 04:58:49.504738 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.504775 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":65,"threads_started":1}
I20260808 04:58:49.504900 16847 consensus_queue.cc:260] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } }
I20260808 04:58:49.505085 16463 heartbeater.cc:503] Master 127.3.174.188:37103 was elected leader, sending a full tablet report...
I20260808 04:58:49.505398 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.505189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.131:54891)
0808 04:58:49.505327 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.505367 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260808 04:58:49.505718 16847 ts_tablet_manager.cc:1428] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:49.505815 16690 heartbeater.cc:503] Master 127.3.174.188:37103 was elected leader, sending a full tablet report...
I20260808 04:58:49.505261 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.505169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.130:39805)
0808 04:58:49.505215 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.505237 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 04:58:49.505973 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.505895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.131:37731)
0808 04:58:49.505925 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.505953 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 04:58:49.506038 16221 heartbeater.cc:503] Master 127.3.174.188:37103 was elected leader, sending a full tablet report...
I20260808 04:58:49.506563 16860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.506273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.129:46173)
0808 04:58:49.506511 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.506539 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
W20260808 04:58:49.539804 16695 tablet.cc:2369] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:49.619261 16855 raft_consensus.cc:482] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:49.619360 16855 raft_consensus.cc:504] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } }
I20260808 04:58:49.619721 16855 leader_election.cc:288] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f0b750dd22c45b79e0b58838242cf2c (127.3.174.129:35607), 6cbd6fbf05d14b67bb4c7601e063004e (127.3.174.131:43685)
I20260808 04:58:49.622377 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.620015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:57463 (local address 127.3.174.131:43685)
0808 04:58:49.620055 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:49.620057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:49.620065 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:49.620105 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:49.620106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:49.620126 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:49.620193 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:49.620356 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.620822 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.621741 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.622084 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.622145 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.622156 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:49.622277 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 04:58:49.622313 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 04:58:49.622394 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.619885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:43685 (local address 127.3.174.130:57463)
0808 04:58:49.619958 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.619970 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:49.620031 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:49.620198 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:49.620199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:49.620202 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:49.620279 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.620280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.620891 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.620892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.621660 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.621662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.622149 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.622150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.622213 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.622216 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:49.622260 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 04:58:49.622299 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 04:58:49.622601 16616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b775f4d8b6f5477da1a2daef0a96c27e" candidate_uuid: "b23c85f2fa75450483b891952ae06569" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" is_pre_election: true
I20260808 04:58:49.622649 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.619956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:48565 (local address 127.3.174.129:35607)
0808 04:58:49.620007 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:49.620009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:49.620258 (+ 249us) server_negotiation.cc:407] Connection header received
0808 04:58:49.620328 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:49.620330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:49.620338 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:49.620379 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:49.620519 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.620990 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.621879 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.622301 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.622363 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.622372 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:49.622549 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 04:58:49.622585 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 04:58:49.622689 16616 raft_consensus.cc:2454] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b23c85f2fa75450483b891952ae06569 in term 0.
I20260808 04:58:49.622874 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.619885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:35607 (local address 127.3.174.130:48565)
0808 04:58:49.620151 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.620162 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:49.620271 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:49.620392 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:49.620393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:49.620394 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:49.620468 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.620470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.621064 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.621065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.621805 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:49.621807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.622397 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.622399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:49.622455 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.622460 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:49.622767 (+ 307us) client_negotiation.cc:240] Negotiation successful
0808 04:58:49.622808 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":62,"thread_start_us":127,"threads_started":1}
I20260808 04:58:49.622864 16267 leader_election.cc:302] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cbd6fbf05d14b67bb4c7601e063004e, b23c85f2fa75450483b891952ae06569; no voters:
I20260808 04:58:49.623026 16855 raft_consensus.cc:2781] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:49.623065 16855 raft_consensus.cc:482] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:49.623083 16855 raft_consensus.cc:3037] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:49.623255 16126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b775f4d8b6f5477da1a2daef0a96c27e" candidate_uuid: "b23c85f2fa75450483b891952ae06569" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f0b750dd22c45b79e0b58838242cf2c" is_pre_election: true
I20260808 04:58:49.623389 16126 raft_consensus.cc:2454] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b23c85f2fa75450483b891952ae06569 in term 0.
I20260808 04:58:49.623911 16855 raft_consensus.cc:504] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } }
I20260808 04:58:49.624027 16855 leader_election.cc:288] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [CANDIDATE]: Term 1 election: Requested vote from peers 4f0b750dd22c45b79e0b58838242cf2c (127.3.174.129:35607), 6cbd6fbf05d14b67bb4c7601e063004e (127.3.174.131:43685)
I20260808 04:58:49.624200 16126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b775f4d8b6f5477da1a2daef0a96c27e" candidate_uuid: "b23c85f2fa75450483b891952ae06569" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f0b750dd22c45b79e0b58838242cf2c"
I20260808 04:58:49.624286 16126 raft_consensus.cc:3037] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:49.624260 16616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b775f4d8b6f5477da1a2daef0a96c27e" candidate_uuid: "b23c85f2fa75450483b891952ae06569" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cbd6fbf05d14b67bb4c7601e063004e"
I20260808 04:58:49.624351 16616 raft_consensus.cc:3037] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:49.624917 16126 raft_consensus.cc:2454] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b23c85f2fa75450483b891952ae06569 in term 1.
I20260808 04:58:49.624948 16616 raft_consensus.cc:2454] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b23c85f2fa75450483b891952ae06569 in term 1.
I20260808 04:58:49.625075 16263 leader_election.cc:302] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f0b750dd22c45b79e0b58838242cf2c, b23c85f2fa75450483b891952ae06569; no voters:
I20260808 04:58:49.625174 16855 raft_consensus.cc:2781] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:49.625339 16855 raft_consensus.cc:686] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 1 LEADER]: Becoming Leader. State: Replica: b23c85f2fa75450483b891952ae06569, State: Running, Role: LEADER
I20260808 04:58:49.625408 16855 consensus_queue.cc:237] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } }
I20260808 04:58:49.625658 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.625564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.130:44917)
0808 04:58:49.625596 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.625632 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260808 04:58:49.626170 15865 catalog_manager.cc:5495] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 reported cstate change: term changed from 0 to 1, leader changed from <none> to b23c85f2fa75450483b891952ae06569 (127.3.174.130). New cstate: current_term: 1 leader_uuid: "b23c85f2fa75450483b891952ae06569" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b23c85f2fa75450483b891952ae06569" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 } health_report { overall_health: UNKNOWN } } }
I20260808 04:58:49.644760 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.642599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.3.174.130:36689)
0808 04:58:49.642656 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:49.642658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:49.642668 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:49.642702 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:49.642703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:49.642719 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:49.642782 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:49.642902 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.643390 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.644099 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:49.644285 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:49.644338 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:49.644477 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:49.644574 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:49.644636 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:49.644672 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 04:58:49.644694 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 04:58:49.645876 16616 raft_consensus.cc:1261] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [term 1 FOLLOWER]: Refusing update from remote peer b23c85f2fa75450483b891952ae06569: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:49.645947 16126 raft_consensus.cc:1261] T b775f4d8b6f5477da1a2daef0a96c27e P 4f0b750dd22c45b79e0b58838242cf2c [term 1 FOLLOWER]: Refusing update from remote peer b23c85f2fa75450483b891952ae06569: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:49.646238 16866 consensus_queue.cc:1035] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cbd6fbf05d14b67bb4c7601e063004e" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:49.646380 16855 consensus_queue.cc:1035] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f0b750dd22c45b79e0b58838242cf2c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:49.647058 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.646942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.131:42123)
0808 04:58:49.646987 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.647021 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 04:58:49.647562 16860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.647460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.129:50779)
0808 04:58:49.647502 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.647529 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 04:58:49.648217 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.648133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.130:33111)
0808 04:58:49.648170 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.648194 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 04:58:49.648975 16860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.648883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.129:43251)
0808 04:58:49.648924 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.648949 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 04:58:49.649065 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.648865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.130:35013)
0808 04:58:49.649010 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.649041 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260808 04:58:49.649430 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:49.649353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35583 (local address 127.3.174.131:53403)
0808 04:58:49.649381 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.649406 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260808 04:58:49.650506 15865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:49.650595 15865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:49.651985 15865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36bd4278aaa94063923d5e6ffddef9e4: Sending DeleteTablet for 3 replicas of tablet b775f4d8b6f5477da1a2daef0a96c27e
I20260808 04:58:49.652275 16093 tablet_service.cc:1514] Processing DeleteTablet for tablet b775f4d8b6f5477da1a2daef0a96c27e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:49 UTC) from {username='oryx', principal='oryx/127.3.174.188@KRBTEST.COM'} at 127.0.0.1:58406
I20260808 04:58:49.652321 16092 tablet_service.cc:1514] Processing DeleteTablet for tablet b775f4d8b6f5477da1a2daef0a96c27e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:49 UTC) from {username='oryx', principal='oryx/127.3.174.188@KRBTEST.COM'} at 127.0.0.1:58406
I20260808 04:58:49.652410 16343 tablet_service.cc:1514] Processing DeleteTablet for tablet b775f4d8b6f5477da1a2daef0a96c27e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:49 UTC) from {username='oryx', principal='oryx/127.3.174.188@KRBTEST.COM'} at 127.0.0.1:52562
I20260808 04:58:49.652580 16901 tablet_replica.cc:330] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: stopping tablet replica
I20260808 04:58:49.652642 16901 raft_consensus.cc:2229] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:49.652714 16590 tablet_service.cc:1514] Processing DeleteTablet for tablet b775f4d8b6f5477da1a2daef0a96c27e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:49 UTC) from {username='oryx', principal='oryx/127.3.174.188@KRBTEST.COM'} at 127.0.0.1:53026
I20260808 04:58:49.652769 16901 raft_consensus.cc:2258] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:49.652956 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 15779
I20260808 04:58:49.653100 16902 tablet_replica.cc:330] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: stopping tablet replica
I20260808 04:58:49.653203 16902 raft_consensus.cc:2229] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:49.653237 16901 ts_tablet_manager.cc:1905] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:49.653569 16902 raft_consensus.cc:2258] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:49.655059 16901 ts_tablet_manager.cc:1918] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:49.655112 16901 log.cc:1196] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/b775f4d8b6f5477da1a2daef0a96c27e
I20260808 04:58:49.655310 16901 ts_tablet_manager.cc:1939] T b775f4d8b6f5477da1a2daef0a96c27e P b23c85f2fa75450483b891952ae06569: Deleting consensus metadata
I20260808 04:58:49.655548 15843 catalog_manager.cc:4841] TS b23c85f2fa75450483b891952ae06569 (127.3.174.130:36689): tablet b775f4d8b6f5477da1a2daef0a96c27e (table test-table [id=f4709e9b396b4a9cb61f5fae95d2ef26]) successfully deleted
I20260808 04:58:49.656167 16902 ts_tablet_manager.cc:1905] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:49.660408 16902 ts_tablet_manager.cc:1918] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:49.660507 16902 log.cc:1196] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/b775f4d8b6f5477da1a2daef0a96c27e
I20260808 04:58:49.660760 16902 ts_tablet_manager.cc:1939] T b775f4d8b6f5477da1a2daef0a96c27e P 6cbd6fbf05d14b67bb4c7601e063004e: Deleting consensus metadata
I20260808 04:58:49.661085 15847 catalog_manager.cc:4841] TS 6cbd6fbf05d14b67bb4c7601e063004e (127.3.174.131:43685): tablet b775f4d8b6f5477da1a2daef0a96c27e (table test-table [id=f4709e9b396b4a9cb61f5fae95d2ef26]) successfully deleted
W20260808 04:58:49.666289 16831 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:49.666130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:35607 (local address 127.0.0.1:58418)
0808 04:58:49.666194 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:49.666231 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:35607: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 04:58:49.666483 15841 catalog_manager.cc:4584] TS 4f0b750dd22c45b79e0b58838242cf2c (127.3.174.129:35607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b775f4d8b6f5477da1a2daef0a96c27e: Network error: Client connection negotiation failed: client connection to 127.3.174.129:35607: connect: Connection reset by peer (error 104)
I20260808 04:58:49.666507 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 16240
I20260808 04:58:49.679167 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 16477
I20260808 04:58:49.690970 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 15233
I20260808 04:58:49.699453 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 15440
2026-08-08T04:58:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4974 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:49 dist-test-slave-gjw7 krb5kdc[16909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:49 dist-test-slave-gjw7 krb5kdc[16909](info): set up 2 sockets
Aug 08 04:58:49 dist-test-slave-gjw7 krb5kdc[16909](info): commencing operation
krb5kdc: starting...
W20260808 04:58:50.882577 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:50 dist-test-slave-gjw7 krb5kdc[16909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:50Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.190.keytab.
Entry for principal oryx/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.190.keytab.
I20260808 04:58:51.003778 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35363
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.190.keytab
--principal=oryx/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:35363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:51.085726 16930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:51.085918 16930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:51.085939 16930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:51.087062 16930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:51.087103 16930 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 04:58:51.087116 16930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:51.087127 16930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:51.087137 16930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:51.087152 16930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:51.088680 16930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:35363
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.190.keytab
--principal=oryx/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:51.088900 16930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:51.089110 16930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:51.093722 16937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:51.093747 16940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:51.093768 16930 server_base.cc:1034] running on GCE node
W20260808 04:58:51.093747 16938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:51.094174 16930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:51.094439 16930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:51.095577 16930 hybrid_clock.cc:648] HybridClock initialized: now 1786165131095557 us; error 37 us; skew 500 ppm
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:51.097997 16930 init.cc:377] Logged in from keytab as oryx/127.3.174.190@KRBTEST.COM (short username oryx)
I20260808 04:58:51.098469 16930 webserver.cc:460] Webserver started at http://127.3.174.190:32791/ using document root <none> and password file <none>
I20260808 04:58:51.098690 16930 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:51.098744 16930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:51.098879 16930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:51.099977 16930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "bceacd09f5ac4db7b25adfab5afc7082"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "71fe72eefe212958e06002052008368b"
server_key_iv: "4b271376d5b8a8fa39097a56af417dc9"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.100330 16930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "bceacd09f5ac4db7b25adfab5afc7082"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "71fe72eefe212958e06002052008368b"
server_key_iv: "4b271376d5b8a8fa39097a56af417dc9"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.101737 16930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 04:58:51.102432 16950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.102593 16930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.102679 16930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "bceacd09f5ac4db7b25adfab5afc7082"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "71fe72eefe212958e06002052008368b"
server_key_iv: "4b271376d5b8a8fa39097a56af417dc9"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.102754 16930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:51.121430 16930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:51.123396 16930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:51.123507 16930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:51.128037 16930 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:35363
I20260808 04:58:51.128111 17063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:35363 every 8 connection(s)
I20260808 04:58:51.128451 16930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:51.128608 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 16930
I20260808 04:58:51.128703 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:51.128957 3770 external_mini_cluster.cc:1428] Setting key 5bd458c4d40b0372ca4a282f0a221ca1
I20260808 04:58:51.129251 17066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:51.132098 17066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082: Bootstrap starting.
I20260808 04:58:51.132707 17066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:51.132978 17066 log.cc:824] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:51.133667 17066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082: No bootstrap required, opened a new log
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:51.135701 17066 raft_consensus.cc:348] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bceacd09f5ac4db7b25adfab5afc7082" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35363 } }
I20260808 04:58:51.135840 17066 raft_consensus.cc:374] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:51.135864 17066 raft_consensus.cc:729] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bceacd09f5ac4db7b25adfab5afc7082, State: Initialized, Role: FOLLOWER
I20260808 04:58:51.135991 17066 consensus_queue.cc:260] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bceacd09f5ac4db7b25adfab5afc7082" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35363 } }
I20260808 04:58:51.136077 17066 raft_consensus.cc:388] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:51.136117 17066 raft_consensus.cc:482] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:51.136169 17066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:51.136767 17066 raft_consensus.cc:504] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bceacd09f5ac4db7b25adfab5afc7082" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35363 } }
I20260808 04:58:51.136860 17066 leader_election.cc:302] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bceacd09f5ac4db7b25adfab5afc7082; no voters:
I20260808 04:58:51.137025 17066 leader_election.cc:288] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:51.137070 17075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:51.137194 17075 raft_consensus.cc:686] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [term 1 LEADER]: Becoming Leader. State: Replica: bceacd09f5ac4db7b25adfab5afc7082, State: Running, Role: LEADER
I20260808 04:58:51.137318 17066 sys_catalog.cc:564] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:51.137279 17075 consensus_queue.cc:237] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bceacd09f5ac4db7b25adfab5afc7082" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35363 } }
I20260808 04:58:51.137697 17078 sys_catalog.cc:455] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bceacd09f5ac4db7b25adfab5afc7082" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bceacd09f5ac4db7b25adfab5afc7082" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35363 } } }
I20260808 04:58:51.137800 17078 sys_catalog.cc:458] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:51.137696 17075 sys_catalog.cc:455] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bceacd09f5ac4db7b25adfab5afc7082. Latest consensus state: current_term: 1 leader_uuid: "bceacd09f5ac4db7b25adfab5afc7082" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bceacd09f5ac4db7b25adfab5afc7082" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35363 } } }
I20260808 04:58:51.138012 17075 sys_catalog.cc:458] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:51.138129 17086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:51.138676 17086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:51.139232 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.129465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.3.174.190:35363)
0808 04:58:51.129658 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.129668 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.129709 (+ 41us) server_negotiation.cc:407] Connection header received
0808 04:58:51.130302 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.130309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.130354 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.130474 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:51.131084 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.131990 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.132810 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.133230 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.133347 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.135678 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:51.135697 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:51.135697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:51.135708 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:51.137448 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.137858 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.137861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.137861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.137965 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.138173 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.138174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.138175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.138482 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:51.138634 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.138952 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.139073 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260808 04:58:51.142604 17086 catalog_manager.cc:1347] Generated new cluster ID: 3ece5bd13c4c4da182e8e5308da1eb4b
I20260808 04:58:51.142648 17086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:51.161321 17086 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:51.161873 17086 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:51.164623 17086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082: Generated new TSK 0
I20260808 04:58:51.164796 17086 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab.
Entry for principal oryx/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab.
I20260808 04:58:51.194051 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35363
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab
--principal=oryx/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:51.305568 17124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:51.305774 17124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:51.305807 17124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:51.307036 17124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:51.307080 17124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:51.307107 17124 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:58:51.307130 17124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:51.308687 17124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab
--principal=oryx/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.174.190:35363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:51.308939 17124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:51.309161 17124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:51.314165 17131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:51.314175 17139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:51.314158 17132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:51.314253 17124 server_base.cc:1034] running on GCE node
I20260808 04:58:51.314647 17124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:51.314914 17124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:51.316071 17124 hybrid_clock.cc:648] HybridClock initialized: now 1786165131316060 us; error 31 us; skew 500 ppm
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:51.318717 17124 init.cc:377] Logged in from keytab as oryx/127.3.174.129@KRBTEST.COM (short username oryx)
I20260808 04:58:51.319173 17124 webserver.cc:460] Webserver started at http://127.3.174.129:44871/ using document root <none> and password file <none>
I20260808 04:58:51.319382 17124 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:51.319423 17124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:51.319517 17124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:51.320552 17124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "87459be0cab84bca91f13272cb87f60c"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "a2d6d0ee6b20c7c3f8be057245f0f397"
server_key_iv: "f1e2e57a85dd3a9d928ff7e4157136e8"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.320906 17124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "87459be0cab84bca91f13272cb87f60c"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "a2d6d0ee6b20c7c3f8be057245f0f397"
server_key_iv: "f1e2e57a85dd3a9d928ff7e4157136e8"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.322182 17124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:51.322844 17155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.323006 17124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:51.323089 17124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "87459be0cab84bca91f13272cb87f60c"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "a2d6d0ee6b20c7c3f8be057245f0f397"
server_key_iv: "f1e2e57a85dd3a9d928ff7e4157136e8"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.323145 17124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:51.342478 17124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:51.344679 17124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:51.344805 17124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:51.345326 17124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:51.345374 17124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.345398 17124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:51.345409 17124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.352232 17124 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:46767
I20260808 04:58:51.352434 17349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:46767 every 8 connection(s)
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:51.352608 17124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:58:51.356073 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.345796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:45929 (local address 127.3.174.190:35363)
0808 04:58:51.345865 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.345868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.346079 (+ 211us) server_negotiation.cc:407] Connection header received
0808 04:58:51.346689 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.346690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.346704 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.346766 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:51.347692 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.348461 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.350058 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.350377 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.350445 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.353154 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:51.353156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:51.353157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:51.353166 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:51.354706 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.355092 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.355093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.355093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.355154 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.355404 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.355405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.355406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.355525 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:51.355582 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.355877 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.355981 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 04:58:51.356261 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.345766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35363 (local address 127.3.174.129:45929)
0808 04:58:51.345957 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.346000 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.346581 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.346883 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.346890 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.347285 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:51.347596 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.347602 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.348568 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.348569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.349963 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.349966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.350480 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.350481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.350615 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.351245 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:51.351257 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:51.353062 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:51.354792 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.354799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.354802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.355022 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.355209 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.355210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.355210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.355340 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.355582 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:51.355590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:51.355727 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.355868 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.356042 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":117,"threads_started":1}
I20260808 04:58:51.356755 17350 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35363
I20260808 04:58:51.356844 17350 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:51.357046 17350 heartbeater.cc:511] Master 127.3.174.190:35363 requested a full tablet report, sending...
I20260808 04:58:51.357618 16976 ts_manager.cc:194] Registered new tserver with Master: 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767)
I20260808 04:58:51.358354 16976 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:45929
I20260808 04:58:51.361039 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 17124
I20260808 04:58:51.361138 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:51.361369 3770 external_mini_cluster.cc:1428] Setting key 88fcfac4410aede9d2942f586fdad9bd
I20260808 04:58:51.363445 17361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.357786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35363 (local address 127.3.174.129:59855)
0808 04:58:51.357908 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.357926 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.357982 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.358129 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.358130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.358312 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:51.358363 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.358365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.359185 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.359185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.359700 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.359702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.360026 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.360027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.360083 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.360427 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:51.360431 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:51.360687 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:51.362404 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.362405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.362405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.362602 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.362794 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.362795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.362795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.362863 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.363131 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:51.363132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:51.363180 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.363244 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.363347 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260808 04:58:51.363732 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.357826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:59855 (local address 127.3.174.190:35363)
0808 04:58:51.357864 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.357866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.357983 (+ 117us) server_negotiation.cc:407] Connection header received
0808 04:58:51.358050 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.358051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.358062 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.358113 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:51.358432 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.359092 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.359789 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.359970 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.360022 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.360757 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:51.360759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:51.360759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:51.360766 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:51.362332 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.362671 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.362672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.362673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.362730 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.362921 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.362922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.362922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.363043 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:51.363099 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.363313 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.363453 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for oryx/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab.
Entry for principal oryx/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab.
I20260808 04:58:51.385818 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35363
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab
--principal=oryx/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:51.468725 17365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:51.468945 17365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:51.468967 17365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:51.470131 17365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:51.470173 17365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:51.470191 17365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:58:51.470208 17365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:51.471761 17365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab
--principal=oryx/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.174.190:35363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:51.472005 17365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:51.472223 17365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:51.477167 17376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:51.477240 17365 server_base.cc:1034] running on GCE node
W20260808 04:58:51.477239 17379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:51.477167 17384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:51.477619 17365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:51.477856 17365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:51.479032 17365 hybrid_clock.cc:648] HybridClock initialized: now 1786165131479016 us; error 28 us; skew 500 ppm
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:51.481597 17365 init.cc:377] Logged in from keytab as oryx/127.3.174.130@KRBTEST.COM (short username oryx)
I20260808 04:58:51.482046 17365 webserver.cc:460] Webserver started at http://127.3.174.130:35811/ using document root <none> and password file <none>
I20260808 04:58:51.482273 17365 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:51.482326 17365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:51.482456 17365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:51.483510 17365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "b7419c18963e4dd39ae31733d1264781"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "1c4ad0abd01a3e91a49435885fa1162c"
server_key_iv: "96df9b20236fd4afe1e945f53522bd59"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.483860 17365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "b7419c18963e4dd39ae31733d1264781"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "1c4ad0abd01a3e91a49435885fa1162c"
server_key_iv: "96df9b20236fd4afe1e945f53522bd59"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.485116 17365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:51.485883 17399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.486042 17365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.486137 17365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "b7419c18963e4dd39ae31733d1264781"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "1c4ad0abd01a3e91a49435885fa1162c"
server_key_iv: "96df9b20236fd4afe1e945f53522bd59"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.486205 17365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:51.504267 17365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:51.506531 17365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:51.506668 17365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:51.507187 17365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:51.507237 17365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.507268 17365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:51.507285 17365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:51.514101 17365 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:43319
I20260808 04:58:51.514209 17614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:43319 every 8 connection(s)
I20260808 04:58:51.514472 17365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:58:51.517066 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.507622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:52579 (local address 127.3.174.190:35363)
0808 04:58:51.507672 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.507675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.507906 (+ 231us) server_negotiation.cc:407] Connection header received
0808 04:58:51.508455 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.508457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.508471 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.508531 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:51.509480 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.510242 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.511708 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.512007 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.512084 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.514574 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:51.514576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:51.514576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:51.514584 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:51.515794 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.516165 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.516166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.516167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.516232 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.516451 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.516452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.516452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.516573 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:51.516629 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.516890 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.516973 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 04:58:51.517221 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.507629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35363 (local address 127.3.174.130:52579)
0808 04:58:51.507783 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.507820 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.508349 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.508681 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.508687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.509060 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:51.509379 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.509387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.510339 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.510341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.511630 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.511633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.512099 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.512100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.512238 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.512703 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:51.512716 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:51.514485 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:51.515869 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.515876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.515878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.516098 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.516286 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.516287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.516287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.516394 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.516628 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:51.516634 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:51.516770 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.516860 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.517011 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":92,"threads_started":1}
I20260808 04:58:51.517684 17615 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35363
I20260808 04:58:51.517761 17615 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:51.517938 17615 heartbeater.cc:511] Master 127.3.174.190:35363 requested a full tablet report, sending...
I20260808 04:58:51.518328 16979 ts_manager.cc:194] Registered new tserver with Master: b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319)
I20260808 04:58:51.518788 16979 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:52579
I20260808 04:58:51.520792 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 17365
I20260808 04:58:51.520908 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:51.521106 3770 external_mini_cluster.cc:1428] Setting key 3660fa81fa3014bb8ebe1fa2758b3c06
I20260808 04:58:51.523801 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.518686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:44891 (local address 127.3.174.190:35363)
0808 04:58:51.518726 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.518727 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.518845 (+ 118us) server_negotiation.cc:407] Connection header received
0808 04:58:51.518886 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.518887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.518897 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.518947 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:51.519407 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.519871 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.520499 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.520663 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.520704 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.521609 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:51.521610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:51.521610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:51.521617 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:51.522789 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.523114 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.523114 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.523114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.523157 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.523301 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.523302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.523302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.523428 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:51.523490 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.523634 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.523703 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 04:58:51.523810 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.518622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35363 (local address 127.3.174.130:44891)
0808 04:58:51.518779 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.518788 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.518840 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.519077 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.519078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.519278 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:51.519331 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.519333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.519950 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.519951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.520442 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.520444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.520723 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.520723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.520793 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.521161 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:51.521166 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:51.521540 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:51.522860 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.522861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.522861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.523051 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.523196 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.523196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.523196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.523252 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.523486 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:51.523487 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:51.523535 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.523604 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.523705 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for oryx/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab.
Entry for principal oryx/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab.
I20260808 04:58:51.547930 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35363
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab
--principal=oryx/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:51.630920 17631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:51.631132 17631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:51.631155 17631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:51.632287 17631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:51.632326 17631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:51.632345 17631 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 04:58:51.632361 17631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:51.633904 17631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab
--principal=oryx/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.174.190:35363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:51.634120 17631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:51.634332 17631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:51.639315 17643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:51.639334 17642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:51.639473 17645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:51.639323 17631 server_base.cc:1034] running on GCE node
I20260808 04:58:51.639708 17631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:51.639930 17631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:51.641081 17631 hybrid_clock.cc:648] HybridClock initialized: now 1786165131641063 us; error 34 us; skew 500 ppm
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:51.643775 17631 init.cc:377] Logged in from keytab as oryx/127.3.174.131@KRBTEST.COM (short username oryx)
I20260808 04:58:51.644227 17631 webserver.cc:460] Webserver started at http://127.3.174.131:35197/ using document root <none> and password file <none>
I20260808 04:58:51.644418 17631 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:51.644465 17631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:51.644570 17631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:51.645705 17631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "db428020657e4f7ba0a2fe5b8edb0629"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "7e7745290a71f6cf8b83721204f799cb"
server_key_iv: "70612e074aa77dcc27c0d419848755f6"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.646091 17631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "db428020657e4f7ba0a2fe5b8edb0629"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "7e7745290a71f6cf8b83721204f799cb"
server_key_iv: "70612e074aa77dcc27c0d419848755f6"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.647526 17631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:51.648243 17658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.648407 17631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:51.648490 17631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "db428020657e4f7ba0a2fe5b8edb0629"
format_stamp: "Formatted at 2026-08-08 04:58:51 on dist-test-slave-gjw7"
server_key: "7e7745290a71f6cf8b83721204f799cb"
server_key_iv: "70612e074aa77dcc27c0d419848755f6"
server_key_version: "encryptionkey@0"
I20260808 04:58:51.648542 17631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:51.666102 17631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:51.668056 17631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:51.668160 17631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:51.668635 17631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:51.668673 17631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.668694 17631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:51.668707 17631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.674862 17631 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:43531
I20260808 04:58:51.674939 17884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:43531 every 8 connection(s)
I20260808 04:58:51.675314 17631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:51.678220 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.669088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:57729 (local address 127.3.174.190:35363)
0808 04:58:51.669164 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.669166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.669404 (+ 238us) server_negotiation.cc:407] Connection header received
0808 04:58:51.670012 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.670014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.670028 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.670104 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:51.670931 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.671660 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.673075 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.673391 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.673468 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.675895 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:51.675897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:51.675897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:51.675914 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:51.677078 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.677520 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.677521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.677521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.677571 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.677748 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.677748 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.677749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.677839 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:51.677888 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.678070 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.678128 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 04:58:51.678414 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.669063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35363 (local address 127.3.174.131:57729)
0808 04:58:51.669242 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.669284 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.669890 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.670181 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.670186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.670527 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:51.670819 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.670827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.671763 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.671764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.672976 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.672979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.673470 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.673472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.673606 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.674098 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:51.674109 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:51.675811 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:51.677154 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.677160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.677162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.677460 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.677620 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.677620 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.677620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.677712 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.677881 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:51.677884 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:51.677984 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.678057 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.678176 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260808 04:58:51.678864 17886 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35363
I20260808 04:58:51.678941 17886 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:51.679105 17886 heartbeater.cc:511] Master 127.3.174.190:35363 requested a full tablet report, sending...
I20260808 04:58:51.679505 16976 ts_manager.cc:194] Registered new tserver with Master: db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531)
I20260808 04:58:51.679975 16976 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:57729
I20260808 04:58:51.683354 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 17631
I20260808 04:58:51.683435 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:51.683591 3770 external_mini_cluster.cc:1428] Setting key 545d6f03205bdce5a1a958382eddb3e1
I20260808 04:58:51.684391 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:58:51.684859 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.679916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:47443 (local address 127.3.174.190:35363)
0808 04:58:51.679956 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.679957 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.680015 (+ 58us) server_negotiation.cc:407] Connection header received
0808 04:58:51.680089 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.680089 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.680098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.680141 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:51.680437 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.680886 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.681609 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.681812 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.681863 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.682585 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:51.682586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:51.682586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:51.682593 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:51.683712 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.684074 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.684075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.684075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.684134 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.684391 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.684392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.684392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.684509 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:51.684568 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.684710 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.684773 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:58:51.684872 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.679845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35363 (local address 127.3.174.131:47443)
0808 04:58:51.679959 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.679969 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.680029 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.680147 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.680147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.680315 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:51.680365 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.680367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.680968 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.680969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.681542 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.681544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.681869 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.681870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.681933 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.682260 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:51.682264 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:51.682507 (+ 243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:51.683803 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.683804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.683805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.684008 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.684197 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:51.684198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:51.684199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:51.684259 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:51.684563 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:51.684564 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:51.684607 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.684673 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.684769 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":63,"threads_started":1}
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 04:58:51.706199 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:51.701157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39912 (local address 127.3.174.190:35363)
0808 04:58:51.701215 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.701217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.701235 (+ 18us) server_negotiation.cc:407] Connection header received
0808 04:58:51.701260 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.701261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.701269 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.701356 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:51.701920 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.702398 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.703106 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.703291 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.703343 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.706119 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39912: BlockingRecv error: recv got EOF from 127.0.0.1:39912 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 04:58:51 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:51.712603 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.706986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.3.174.190:35363)
0808 04:58:51.707036 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.707038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.707049 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:58:51.707106 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.707107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.707114 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.707166 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:51.707572 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.708014 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.708590 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.708753 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.708799 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.710650 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:51.710652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:51.710652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:51.710658 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:51.711804 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.712089 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.712090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.712091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.712135 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:51.712261 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:51.712262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:51.712262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:51.712348 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:51.712413 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.712487 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.712539 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 04:58:51.714183 16976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:51.715735 16976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:51.722424 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.718462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.3.174.131:43531)
0808 04:58:51.718645 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.718648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.718815 (+ 167us) server_negotiation.cc:407] Connection header received
0808 04:58:51.718954 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.718956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.718974 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.719048 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:51.719256 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.720284 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.721285 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.722021 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.722091 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.722167 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.722294 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.722332 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260808 04:58:51.722415 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.718131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:43531 (local address 127.0.0.1:51986)
0808 04:58:51.718654 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.718665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.718849 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.719112 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.719113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.719115 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:51.719181 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.719183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.720352 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.720353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.721208 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.721211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.722112 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.722113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.722182 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.722198 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.722280 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.722333 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":77,"threads_started":1}
I20260808 04:58:51.722883 17918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.718129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:43319 (local address 127.0.0.1:33024)
0808 04:58:51.718272 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.718310 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.718372 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.718707 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.718711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.718713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:51.718794 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.718796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.720048 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.720050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.721371 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.721375 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.722241 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.722242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.722297 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.722301 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.722353 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.722386 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":79,"thread_start_us":41,"threads_started":1}
I20260808 04:58:51.723130 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.718251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.3.174.130:43319)
0808 04:58:51.718447 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.718452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.718464 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:58:51.718509 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.718511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.718554 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.718696 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:51.718884 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.719953 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.721453 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.722124 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.722188 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.722246 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.722981 (+ 735us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.723029 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":89,"threads_started":1}
I20260808 04:58:51.723297 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.718321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:46767 (local address 127.0.0.1:57762)
0808 04:58:51.718751 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.718761 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.718813 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.719063 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.719064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.719066 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:51.719109 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.719111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.720031 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.720032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.721372 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.721375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.723074 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.723076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.723141 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.723145 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.723190 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.723228 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":35,"thread_start_us":71,"threads_started":1}
I20260808 04:58:51.723227 17758 tablet_service.cc:1467] Processing CreateTablet for tablet 46d3a8e5439f444c9f880d77ee66fe5f (DEFAULT_TABLE table=test-table [id=5eae84b98f3648cb87406abba4976c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:51.723574 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.718432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.3.174.129:46767)
0808 04:58:51.718641 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.718647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.718873 (+ 226us) server_negotiation.cc:407] Connection header received
0808 04:58:51.718915 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.718918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.718933 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.718998 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:51.719193 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.719939 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.722283 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.722904 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.722967 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.723040 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.723439 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.723481 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260808 04:58:51.723646 17758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d3a8e5439f444c9f880d77ee66fe5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:51.723773 17484 tablet_service.cc:1467] Processing CreateTablet for tablet 46d3a8e5439f444c9f880d77ee66fe5f (DEFAULT_TABLE table=test-table [id=5eae84b98f3648cb87406abba4976c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:51.724121 17484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d3a8e5439f444c9f880d77ee66fe5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:51.724138 17236 tablet_service.cc:1467] Processing CreateTablet for tablet 46d3a8e5439f444c9f880d77ee66fe5f (DEFAULT_TABLE table=test-table [id=5eae84b98f3648cb87406abba4976c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:51.724447 17236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d3a8e5439f444c9f880d77ee66fe5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:51.727655 17937 tablet_bootstrap.cc:492] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: Bootstrap starting.
I20260808 04:58:51.727978 17939 tablet_bootstrap.cc:492] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: Bootstrap starting.
I20260808 04:58:51.728386 17937 tablet_bootstrap.cc:654] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:51.728550 17939 tablet_bootstrap.cc:654] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:51.728734 17937 log.cc:824] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:51.728844 17939 log.cc:824] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:51.729074 17938 tablet_bootstrap.cc:492] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: Bootstrap starting.
I20260808 04:58:51.729573 17937 tablet_bootstrap.cc:492] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: No bootstrap required, opened a new log
I20260808 04:58:51.729573 17939 tablet_bootstrap.cc:492] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: No bootstrap required, opened a new log
I20260808 04:58:51.729645 17939 ts_tablet_manager.cc:1397] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:51.729645 17937 ts_tablet_manager.cc:1397] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:51.731524 17939 raft_consensus.cc:348] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } }
I20260808 04:58:51.731524 17937 raft_consensus.cc:348] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } }
I20260808 04:58:51.731694 17939 raft_consensus.cc:374] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:51.731693 17937 raft_consensus.cc:374] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:51.731730 17939 raft_consensus.cc:729] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7419c18963e4dd39ae31733d1264781, State: Initialized, Role: FOLLOWER
I20260808 04:58:51.731740 17937 raft_consensus.cc:729] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87459be0cab84bca91f13272cb87f60c, State: Initialized, Role: FOLLOWER
I20260808 04:58:51.731868 17937 consensus_queue.cc:260] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } }
I20260808 04:58:51.731868 17939 consensus_queue.cc:260] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } }
I20260808 04:58:51.732144 17939 ts_tablet_manager.cc:1428] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 04:58:51.732144 17937 ts_tablet_manager.cc:1428] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 04:58:51.732234 17350 heartbeater.cc:503] Master 127.3.174.190:35363 was elected leader, sending a full tablet report...
I20260808 04:58:51.732276 17615 heartbeater.cc:503] Master 127.3.174.190:35363 was elected leader, sending a full tablet report...
I20260808 04:58:51.732659 17938 tablet_bootstrap.cc:654] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:51.732990 17938 log.cc:824] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:51.733681 17938 tablet_bootstrap.cc:492] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: No bootstrap required, opened a new log
I20260808 04:58:51.733754 17938 ts_tablet_manager.cc:1397] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 04:58:51.735046 17938 raft_consensus.cc:348] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } }
I20260808 04:58:51.735164 17938 raft_consensus.cc:374] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:51.735184 17938 raft_consensus.cc:729] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db428020657e4f7ba0a2fe5b8edb0629, State: Initialized, Role: FOLLOWER
I20260808 04:58:51.735294 17938 consensus_queue.cc:260] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } }
I20260808 04:58:51.735530 17938 ts_tablet_manager.cc:1428] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:58:51.735569 17886 heartbeater.cc:503] Master 127.3.174.190:35363 was elected leader, sending a full tablet report...
W20260808 04:58:51.765110 17616 tablet.cc:2369] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:51.773124 17946 raft_consensus.cc:482] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:51.773239 17946 raft_consensus.cc:504] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } }
I20260808 04:58:51.773630 17946 leader_election.cc:288] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531), 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767)
I20260808 04:58:51.776764 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.773796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:43531 (local address 127.3.174.130:42471)
0808 04:58:51.773854 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.773866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.773917 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.774029 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.774030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.774031 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:51.774103 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.774104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.774705 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.774706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.775881 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.775884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.776545 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.776546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.776609 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.776613 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.776648 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.776690 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 04:58:51.776773 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.773865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:42471 (local address 127.3.174.131:43531)
0808 04:58:51.773920 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.773922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.773931 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:51.773963 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.773964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.773972 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.774021 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:51.774159 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.774632 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.775971 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.776466 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.776538 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.776547 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.776668 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.776710 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 04:58:51.776917 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.773898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:46767 (local address 127.3.174.130:36901)
0808 04:58:51.774085 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:51.774093 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:51.774146 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:51.774289 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:51.774290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:51.774292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:51.774365 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.774366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.774988 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.774989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.776043 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:51.776045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.776695 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.776696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:51.776757 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.776760 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:58:51.776811 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 04:58:51.776842 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260808 04:58:51.776923 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.773998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:36901 (local address 127.3.174.129:46767)
0808 04:58:51.774041 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.774043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.774145 (+ 102us) server_negotiation.cc:407] Connection header received
0808 04:58:51.774197 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.774198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.774207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.774254 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:51.774422 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.774924 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.776112 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.776617 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.776677 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.776685 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.776824 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.776862 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 04:58:51.777072 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d3a8e5439f444c9f880d77ee66fe5f" candidate_uuid: "b7419c18963e4dd39ae31733d1264781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db428020657e4f7ba0a2fe5b8edb0629" is_pre_election: true
I20260808 04:58:51.777196 17794 raft_consensus.cc:2454] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7419c18963e4dd39ae31733d1264781 in term 0.
I20260808 04:58:51.777410 17403 leader_election.cc:302] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7419c18963e4dd39ae31733d1264781, db428020657e4f7ba0a2fe5b8edb0629; no voters:
I20260808 04:58:51.777523 17946 raft_consensus.cc:2781] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:51.777486 17267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d3a8e5439f444c9f880d77ee66fe5f" candidate_uuid: "b7419c18963e4dd39ae31733d1264781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87459be0cab84bca91f13272cb87f60c" is_pre_election: true
I20260808 04:58:51.777573 17946 raft_consensus.cc:482] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:51.777588 17946 raft_consensus.cc:3037] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:51.777632 17267 raft_consensus.cc:2454] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7419c18963e4dd39ae31733d1264781 in term 0.
I20260808 04:58:51.778580 17946 raft_consensus.cc:504] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } }
I20260808 04:58:51.778697 17946 leader_election.cc:288] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [CANDIDATE]: Term 1 election: Requested vote from peers db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531), 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767)
I20260808 04:58:51.778869 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d3a8e5439f444c9f880d77ee66fe5f" candidate_uuid: "b7419c18963e4dd39ae31733d1264781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db428020657e4f7ba0a2fe5b8edb0629"
I20260808 04:58:51.778923 17267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d3a8e5439f444c9f880d77ee66fe5f" candidate_uuid: "b7419c18963e4dd39ae31733d1264781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87459be0cab84bca91f13272cb87f60c"
I20260808 04:58:51.778960 17794 raft_consensus.cc:3037] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:51.779018 17267 raft_consensus.cc:3037] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:51.779625 17794 raft_consensus.cc:2454] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7419c18963e4dd39ae31733d1264781 in term 1.
I20260808 04:58:51.779824 17403 leader_election.cc:302] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7419c18963e4dd39ae31733d1264781, db428020657e4f7ba0a2fe5b8edb0629; no voters:
I20260808 04:58:51.779942 17946 raft_consensus.cc:2781] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:51.779951 17267 raft_consensus.cc:2454] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7419c18963e4dd39ae31733d1264781 in term 1.
I20260808 04:58:51.780112 17946 raft_consensus.cc:686] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 1 LEADER]: Becoming Leader. State: Replica: b7419c18963e4dd39ae31733d1264781, State: Running, Role: LEADER
I20260808 04:58:51.780210 17946 consensus_queue.cc:237] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } }
I20260808 04:58:51.780988 16976 catalog_manager.cc:5495] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7419c18963e4dd39ae31733d1264781 (127.3.174.130). New cstate: current_term: 1 leader_uuid: "b7419c18963e4dd39ae31733d1264781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260808 04:58:51.793996 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:51.791686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.3.174.130:43319)
0808 04:58:51.791739 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:51.791740 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:51.791758 (+ 18us) server_negotiation.cc:407] Connection header received
0808 04:58:51.791846 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:51.791847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:51.791865 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:51.791926 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:51.792066 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.792555 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.793315 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:51.793493 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:51.793553 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:51.793707 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:51.793800 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:51.793869 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:51.793904 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 04:58:51.793922 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 04:58:51.795624 17794 raft_consensus.cc:1261] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Refusing update from remote peer b7419c18963e4dd39ae31733d1264781: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:51.795624 17267 raft_consensus.cc:1261] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Refusing update from remote peer b7419c18963e4dd39ae31733d1264781: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:51.795940 17946 consensus_queue.cc:1035] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:51.796075 17956 consensus_queue.cc:1035] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:51.798811 17961 mvcc.cc:205] Tried to move back new op lower bound from 7316132379832356864 to 7316132379772235776. Current Snapshot: MvccSnapshot[applied={T|T < 7316132379832356864}]
I20260808 04:58:51.800871 16979 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:51.800954 16979 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:51.801684 16979 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082: Sending DeleteTablet for 3 replicas of tablet 46d3a8e5439f444c9f880d77ee66fe5f
I20260808 04:58:51.802031 17758 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d3a8e5439f444c9f880d77ee66fe5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:51 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:51986
I20260808 04:58:51.802037 17236 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d3a8e5439f444c9f880d77ee66fe5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:51 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:57762
I20260808 04:58:51.802266 17978 tablet_replica.cc:330] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: stopping tablet replica
I20260808 04:58:51.802259 17484 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d3a8e5439f444c9f880d77ee66fe5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:51 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:33024
I20260808 04:58:51.802369 17978 raft_consensus.cc:2229] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:51.802476 17978 raft_consensus.cc:2258] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:51.802457 16979 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 04:58:51.802489 17979 tablet_replica.cc:330] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: stopping tablet replica
W20260808 04:58:51.802604 16979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:51.802783 17979 raft_consensus.cc:2229] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:51.802940 17979 raft_consensus.cc:2258] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:51.803169 17978 ts_tablet_manager.cc:1905] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:51.803712 17979 ts_tablet_manager.cc:1905] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:51.804023 17758 tablet_service.cc:1467] Processing CreateTablet for tablet 14bf2b9393394e298b4c816901e27f6b (DEFAULT_TABLE table=test-table [id=640cff6becd345d1a94d6aaa9dd95b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:51.804032 17236 tablet_service.cc:1467] Processing CreateTablet for tablet 14bf2b9393394e298b4c816901e27f6b (DEFAULT_TABLE table=test-table [id=640cff6becd345d1a94d6aaa9dd95b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:51.804129 17758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bf2b9393394e298b4c816901e27f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:51.804143 17236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bf2b9393394e298b4c816901e27f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:51.805408 17979 ts_tablet_manager.cc:1918] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:51.805469 17979 log.cc:1196] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/46d3a8e5439f444c9f880d77ee66fe5f
I20260808 04:58:51.805783 17937 tablet_bootstrap.cc:492] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: Bootstrap starting.
I20260808 04:58:51.805822 17979 ts_tablet_manager.cc:1939] T 46d3a8e5439f444c9f880d77ee66fe5f P b7419c18963e4dd39ae31733d1264781: Deleting consensus metadata
I20260808 04:58:51.806195 17937 tablet_bootstrap.cc:654] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:51.806520 17938 tablet_bootstrap.cc:492] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: Bootstrap starting.
I20260808 04:58:51.806921 17938 tablet_bootstrap.cc:654] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:51.807438 16954 catalog_manager.cc:4841] TS b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319): tablet 46d3a8e5439f444c9f880d77ee66fe5f (table test-table [id=5eae84b98f3648cb87406abba4976c64]) successfully deleted
I20260808 04:58:51.807718 17938 tablet_bootstrap.cc:492] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: No bootstrap required, opened a new log
I20260808 04:58:51.807713 17484 tablet_service.cc:1467] Processing CreateTablet for tablet 14bf2b9393394e298b4c816901e27f6b (DEFAULT_TABLE table=test-table [id=640cff6becd345d1a94d6aaa9dd95b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:51.807776 17938 ts_tablet_manager.cc:1397] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:51.807824 17484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bf2b9393394e298b4c816901e27f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:51.807934 17938 raft_consensus.cc:348] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:51.807998 17938 raft_consensus.cc:374] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:51.808019 17938 raft_consensus.cc:729] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db428020657e4f7ba0a2fe5b8edb0629, State: Initialized, Role: FOLLOWER
I20260808 04:58:51.808068 17938 consensus_queue.cc:260] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:51.808169 17938 ts_tablet_manager.cc:1428] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.809007 17939 tablet_bootstrap.cc:492] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: Bootstrap starting.
I20260808 04:58:51.809401 17937 tablet_bootstrap.cc:492] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: No bootstrap required, opened a new log
I20260808 04:58:51.809442 17937 ts_tablet_manager.cc:1397] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 04:58:51.809558 17939 tablet_bootstrap.cc:654] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:51.809582 17937 raft_consensus.cc:348] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:51.809643 17937 raft_consensus.cc:374] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:51.809661 17937 raft_consensus.cc:729] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87459be0cab84bca91f13272cb87f60c, State: Initialized, Role: FOLLOWER
I20260808 04:58:51.809705 17937 consensus_queue.cc:260] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:51.809801 17937 ts_tablet_manager.cc:1428] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.811125 17978 ts_tablet_manager.cc:1918] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:51.811204 17978 log.cc:1196] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/46d3a8e5439f444c9f880d77ee66fe5f
I20260808 04:58:51.811434 17978 ts_tablet_manager.cc:1939] T 46d3a8e5439f444c9f880d77ee66fe5f P db428020657e4f7ba0a2fe5b8edb0629: Deleting consensus metadata
I20260808 04:58:51.811822 17976 tablet_replica.cc:330] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: stopping tablet replica
I20260808 04:58:51.811928 17976 raft_consensus.cc:2229] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:51.811910 16958 catalog_manager.cc:4841] TS db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531): tablet 46d3a8e5439f444c9f880d77ee66fe5f (table test-table [id=5eae84b98f3648cb87406abba4976c64]) successfully deleted
I20260808 04:58:51.812021 17976 raft_consensus.cc:2258] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:51.812588 17976 ts_tablet_manager.cc:1905] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:51.812639 17939 tablet_bootstrap.cc:492] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: No bootstrap required, opened a new log
I20260808 04:58:51.812678 17939 ts_tablet_manager.cc:1397] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 04:58:51.812825 17939 raft_consensus.cc:348] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:51.812878 17939 raft_consensus.cc:374] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:51.812896 17939 raft_consensus.cc:729] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7419c18963e4dd39ae31733d1264781, State: Initialized, Role: FOLLOWER
I20260808 04:58:51.812930 17939 consensus_queue.cc:260] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:51.813014 17939 ts_tablet_manager.cc:1428] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:51.814008 17976 ts_tablet_manager.cc:1918] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:51.814083 17976 log.cc:1196] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/46d3a8e5439f444c9f880d77ee66fe5f
I20260808 04:58:51.814316 17976 ts_tablet_manager.cc:1939] T 46d3a8e5439f444c9f880d77ee66fe5f P 87459be0cab84bca91f13272cb87f60c: Deleting consensus metadata
I20260808 04:58:51.814674 16958 catalog_manager.cc:4841] TS 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767): tablet 46d3a8e5439f444c9f880d77ee66fe5f (table test-table [id=5eae84b98f3648cb87406abba4976c64]) successfully deleted
W20260808 04:58:51.853369 17351 tablet.cc:2369] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:58:51.925858 17888 tablet.cc:2369] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:58:52.006302 17948 raft_consensus.cc:482] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:52.006412 17948 raft_consensus.cc:504] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.006754 17948 leader_election.cc:288] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319), db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531)
I20260808 04:58:52.009697 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.007076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:34735 (local address 127.3.174.130:43319)
0808 04:58:52.007126 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.007127 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.007163 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:58:52.007307 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.007308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.007321 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.007351 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:52.007531 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.007984 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.008986 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.009375 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.009441 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.009450 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.009585 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.009622 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:58:52.009698 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.007019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:40163 (local address 127.3.174.131:43531)
0808 04:58:52.007086 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.007088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.007156 (+ 68us) server_negotiation.cc:407] Connection header received
0808 04:58:52.007251 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.007252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.007279 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.007337 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:52.007474 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.007979 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.008984 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.009373 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.009440 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.009450 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.009589 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.009623 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:58:52.009728 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.006936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:43319 (local address 127.3.174.129:34735)
0808 04:58:52.007090 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.007115 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.007197 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.007414 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.007415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.007417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:52.007477 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.007478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.008058 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.008059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.008900 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.008902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.009448 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.009449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.009522 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.009526 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.009572 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.009618 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":53,"thread_start_us":62,"threads_started":1}
I20260808 04:58:52.009735 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.006936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:43531 (local address 127.3.174.129:40163)
0808 04:58:52.007078 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.007092 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.007179 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.007344 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.007345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.007347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:52.007413 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.007415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.008067 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.008068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.008897 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.008900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.009450 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.009451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.009523 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.009528 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.009576 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.009624 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":32,"thread_start_us":83,"threads_started":1}
I20260808 04:58:52.009997 17526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bf2b9393394e298b4c816901e27f6b" candidate_uuid: "87459be0cab84bca91f13272cb87f60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7419c18963e4dd39ae31733d1264781" is_pre_election: true
I20260808 04:58:52.010152 17526 raft_consensus.cc:2454] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87459be0cab84bca91f13272cb87f60c in term 0.
I20260808 04:58:52.010238 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bf2b9393394e298b4c816901e27f6b" candidate_uuid: "87459be0cab84bca91f13272cb87f60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db428020657e4f7ba0a2fe5b8edb0629" is_pre_election: true
I20260808 04:58:52.010326 17794 raft_consensus.cc:2454] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87459be0cab84bca91f13272cb87f60c in term 0.
I20260808 04:58:52.010340 17156 leader_election.cc:302] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87459be0cab84bca91f13272cb87f60c, b7419c18963e4dd39ae31733d1264781; no voters:
I20260808 04:58:52.010470 17948 raft_consensus.cc:2781] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:52.010505 17948 raft_consensus.cc:482] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:52.010524 17948 raft_consensus.cc:3037] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.011191 17948 raft_consensus.cc:504] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.011332 17948 leader_election.cc:288] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [CANDIDATE]: Term 1 election: Requested vote from peers b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319), db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531)
I20260808 04:58:52.011492 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bf2b9393394e298b4c816901e27f6b" candidate_uuid: "87459be0cab84bca91f13272cb87f60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db428020657e4f7ba0a2fe5b8edb0629"
I20260808 04:58:52.011564 17794 raft_consensus.cc:3037] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.011523 17526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bf2b9393394e298b4c816901e27f6b" candidate_uuid: "87459be0cab84bca91f13272cb87f60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7419c18963e4dd39ae31733d1264781"
I20260808 04:58:52.011617 17526 raft_consensus.cc:3037] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.012144 17794 raft_consensus.cc:2454] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87459be0cab84bca91f13272cb87f60c in term 1.
I20260808 04:58:52.012323 17159 leader_election.cc:302] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87459be0cab84bca91f13272cb87f60c, db428020657e4f7ba0a2fe5b8edb0629; no voters:
I20260808 04:58:52.012382 17526 raft_consensus.cc:2454] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87459be0cab84bca91f13272cb87f60c in term 1.
I20260808 04:58:52.012425 17948 raft_consensus.cc:2781] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:52.012559 17948 raft_consensus.cc:686] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 1 LEADER]: Becoming Leader. State: Replica: 87459be0cab84bca91f13272cb87f60c, State: Running, Role: LEADER
I20260808 04:58:52.012669 17948 consensus_queue.cc:237] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.013455 16979 catalog_manager.cc:5495] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c reported cstate change: term changed from 0 to 1, leader changed from <none> to 87459be0cab84bca91f13272cb87f60c (127.3.174.129). New cstate: current_term: 1 leader_uuid: "87459be0cab84bca91f13272cb87f60c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } health_report { overall_health: UNKNOWN } } }
Aug 08 04:58:52 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165131, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.190@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:52.060997 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.054086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.3.174.190:35363)
0808 04:58:52.054152 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.054154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.054285 (+ 131us) server_negotiation.cc:407] Connection header received
0808 04:58:52.054364 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.054365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.054374 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.054416 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:52.054919 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.055438 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.056138 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.056337 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.056386 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.058783 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:52.058785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:52.058785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:52.058792 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:52.060029 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:52.060392 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:52.060393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:52.060393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:52.060442 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:52.060593 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:52.060593 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:52.060593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:52.060685 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:52.060730 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.060856 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.060927 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 04:58:52.061102 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.054017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35363 (local address 127.0.0.1:39932)
0808 04:58:52.054208 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.054222 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.054298 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.054464 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.054465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.054794 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:52.054851 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.054854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.055510 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.055511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.056058 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.056060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.056434 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.056435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.056488 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.057220 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:52.057226 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:52.058704 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:52.060110 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:52.060118 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:52.060120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:52.060328 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:52.060492 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:52.060492 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:52.060493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:52.060550 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:52.060732 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:52.060737 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:52.060778 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.060837 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.060993 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":63,"thread_start_us":91,"threads_started":1}
I20260808 04:58:52.063217 16979 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:39932:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 04:58:52.063416 16979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:52.064751 17236 tablet_service.cc:1467] Processing CreateTablet for tablet b126e8e022f049e49423cc4f713efcb1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c6656e4c94c4712ad4080688e092435]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:58:52.064780 17484 tablet_service.cc:1467] Processing CreateTablet for tablet b126e8e022f049e49423cc4f713efcb1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c6656e4c94c4712ad4080688e092435]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:58:52.064921 17484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b126e8e022f049e49423cc4f713efcb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:52.064929 17236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b126e8e022f049e49423cc4f713efcb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:52.066088 17939 tablet_bootstrap.cc:492] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: Bootstrap starting.
I20260808 04:58:52.066179 17758 tablet_service.cc:1467] Processing CreateTablet for tablet b126e8e022f049e49423cc4f713efcb1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c6656e4c94c4712ad4080688e092435]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 04:58:52.066331 17758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b126e8e022f049e49423cc4f713efcb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:52.066540 17939 tablet_bootstrap.cc:654] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:52.067066 17939 tablet_bootstrap.cc:492] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: No bootstrap required, opened a new log
I20260808 04:58:52.067121 17939 ts_tablet_manager.cc:1397] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:52.067233 17939 raft_consensus.cc:348] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.067312 17939 raft_consensus.cc:374] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:52.067337 17939 raft_consensus.cc:729] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7419c18963e4dd39ae31733d1264781, State: Initialized, Role: FOLLOWER
I20260808 04:58:52.067385 17939 consensus_queue.cc:260] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.067479 17939 ts_tablet_manager.cc:1428] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:52.067607 17937 tablet_bootstrap.cc:492] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: Bootstrap starting.
I20260808 04:58:52.067576 17956 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.067646 17956 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:52.067993 17937 tablet_bootstrap.cc:654] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:52.068667 17937 tablet_bootstrap.cc:492] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: No bootstrap required, opened a new log
I20260808 04:58:52.068710 17937 ts_tablet_manager.cc:1397] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:52.068846 17937 raft_consensus.cc:348] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.068902 17937 raft_consensus.cc:374] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:52.068918 17937 raft_consensus.cc:729] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87459be0cab84bca91f13272cb87f60c, State: Initialized, Role: FOLLOWER
I20260808 04:58:52.068958 17937 consensus_queue.cc:260] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.069042 17937 ts_tablet_manager.cc:1428] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:52.069100 17948 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.069247 17948 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:52.069331 17938 tablet_bootstrap.cc:492] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: Bootstrap starting.
I20260808 04:58:52.069718 17938 tablet_bootstrap.cc:654] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:52.070242 17938 tablet_bootstrap.cc:492] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: No bootstrap required, opened a new log
I20260808 04:58:52.070278 17938 ts_tablet_manager.cc:1397] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:52.070412 17938 raft_consensus.cc:348] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.070483 17938 raft_consensus.cc:374] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:52.070502 17938 raft_consensus.cc:729] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db428020657e4f7ba0a2fe5b8edb0629, State: Initialized, Role: FOLLOWER
I20260808 04:58:52.070550 17938 consensus_queue.cc:260] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.070636 17938 ts_tablet_manager.cc:1428] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:52.070761 17952 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.070840 17952 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:52.356346 17956 raft_consensus.cc:482] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:52.356439 17956 raft_consensus.cc:504] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.356585 17956 leader_election.cc:288] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767), db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531)
I20260808 04:58:52.356792 17267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b126e8e022f049e49423cc4f713efcb1" candidate_uuid: "b7419c18963e4dd39ae31733d1264781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87459be0cab84bca91f13272cb87f60c" is_pre_election: true
I20260808 04:58:52.356839 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b126e8e022f049e49423cc4f713efcb1" candidate_uuid: "b7419c18963e4dd39ae31733d1264781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db428020657e4f7ba0a2fe5b8edb0629" is_pre_election: true
I20260808 04:58:52.356909 17267 raft_consensus.cc:2454] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7419c18963e4dd39ae31733d1264781 in term 0.
I20260808 04:58:52.356920 17794 raft_consensus.cc:2454] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7419c18963e4dd39ae31733d1264781 in term 0.
I20260808 04:58:52.357065 17403 leader_election.cc:302] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87459be0cab84bca91f13272cb87f60c, b7419c18963e4dd39ae31733d1264781; no voters:
I20260808 04:58:52.357160 17956 raft_consensus.cc:2781] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:52.357203 17956 raft_consensus.cc:482] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:52.357221 17956 raft_consensus.cc:3037] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.358052 17956 raft_consensus.cc:504] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.358172 17956 leader_election.cc:288] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [CANDIDATE]: Term 1 election: Requested vote from peers 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767), db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531)
I20260808 04:58:52.358362 17267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b126e8e022f049e49423cc4f713efcb1" candidate_uuid: "b7419c18963e4dd39ae31733d1264781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87459be0cab84bca91f13272cb87f60c"
I20260808 04:58:52.358397 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b126e8e022f049e49423cc4f713efcb1" candidate_uuid: "b7419c18963e4dd39ae31733d1264781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db428020657e4f7ba0a2fe5b8edb0629"
I20260808 04:58:52.358448 17267 raft_consensus.cc:3037] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.358464 17794 raft_consensus.cc:3037] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.359025 17794 raft_consensus.cc:2454] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7419c18963e4dd39ae31733d1264781 in term 1.
I20260808 04:58:52.359126 17267 raft_consensus.cc:2454] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7419c18963e4dd39ae31733d1264781 in term 1.
I20260808 04:58:52.359189 17403 leader_election.cc:302] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7419c18963e4dd39ae31733d1264781, db428020657e4f7ba0a2fe5b8edb0629; no voters:
I20260808 04:58:52.359285 17956 raft_consensus.cc:2781] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:52.359390 17956 raft_consensus.cc:686] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [term 1 LEADER]: Becoming Leader. State: Replica: b7419c18963e4dd39ae31733d1264781, State: Running, Role: LEADER
I20260808 04:58:52.359433 17956 consensus_queue.cc:237] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.359678 18011 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: TxnStatusTablet state changed. Reason: New leader b7419c18963e4dd39ae31733d1264781. Latest consensus state: current_term: 1 leader_uuid: "b7419c18963e4dd39ae31733d1264781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.359764 18011 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:58:52.359907 18014 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 04:58:52.360019 18014 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 04:58:52.360042 16979 catalog_manager.cc:5495] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7419c18963e4dd39ae31733d1264781 (127.3.174.130). New cstate: current_term: 1 leader_uuid: "b7419c18963e4dd39ae31733d1264781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } health_report { overall_health: UNKNOWN } } }
I20260808 04:58:52.421326 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.419004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.3.174.130:43319)
0808 04:58:52.419060 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.419063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.419073 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:58:52.419137 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.419138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.419150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.419211 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:52.419331 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.419807 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.420564 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.420768 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.420834 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.421010 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.421051 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.421110 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.421191 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.421213 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 04:58:52.421330 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.418919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:43319 (local address 127.0.0.1:33040)
0808 04:58:52.418999 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.419013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.419066 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.419218 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.419218 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.419220 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:52.419277 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.419280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.419893 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.419894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.420474 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.420477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.420840 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.420841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.420909 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.420928 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.421118 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.421126 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.421175 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.421229 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 04:58:52.422569 17267 raft_consensus.cc:1261] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Refusing update from remote peer b7419c18963e4dd39ae31733d1264781: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:52.422607 17794 raft_consensus.cc:1261] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Refusing update from remote peer b7419c18963e4dd39ae31733d1264781: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:52.422847 17956 consensus_queue.cc:1035] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:52.422982 18011 consensus_queue.cc:1035] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:52.423524 17952 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: TxnStatusTablet state changed. Reason: New leader b7419c18963e4dd39ae31733d1264781. Latest consensus state: current_term: 1 leader_uuid: "b7419c18963e4dd39ae31733d1264781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.423624 17952 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:52.424044 17948 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: TxnStatusTablet state changed. Reason: New leader b7419c18963e4dd39ae31733d1264781. Latest consensus state: current_term: 1 leader_uuid: "b7419c18963e4dd39ae31733d1264781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.424137 17948 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:52.424274 17956 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7419c18963e4dd39ae31733d1264781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.424365 17956 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:58:52.424432 18020 mvcc.cc:205] Tried to move back new op lower bound from 7316132382400946176 to 7316132382144499712. Current Snapshot: MvccSnapshot[applied={T|T < 7316132382400946176}]
I20260808 04:58:52.424741 17948 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7419c18963e4dd39ae31733d1264781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.424827 17948 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P 87459be0cab84bca91f13272cb87f60c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:52.425707 18011 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7419c18963e4dd39ae31733d1264781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.425786 18011 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P b7419c18963e4dd39ae31733d1264781: This TxnStatusTablet replica's current role is: LEADER
I20260808 04:58:52.426195 17952 tablet_replica.cc:439] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7419c18963e4dd39ae31733d1264781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } } }
I20260808 04:58:52.426281 17952 tablet_replica.cc:442] T b126e8e022f049e49423cc4f713efcb1 P db428020657e4f7ba0a2fe5b8edb0629: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 04:58:52.426221 18022 mvcc.cc:205] Tried to move back new op lower bound from 7316132382400946176 to 7316132382144499712. Current Snapshot: MvccSnapshot[applied={T|T < 7316132382400946176}]
I20260808 04:58:52.429946 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.427157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.3.174.129:46767)
0808 04:58:52.427259 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.427261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.427270 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:52.427317 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.427318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.427337 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.427406 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:52.427562 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.428031 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.429118 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.429400 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.429460 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.429627 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.429737 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.429792 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.429847 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.429864 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":49,"threads_started":1}
I20260808 04:58:52.434756 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.432518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:45227 (local address 127.3.174.130:43319)
0808 04:58:52.432565 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.432566 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.432660 (+ 94us) server_negotiation.cc:407] Connection header received
0808 04:58:52.432735 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.432735 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.432746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.432790 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:52.432904 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.433394 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.434129 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.434309 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.434368 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.434528 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.434556 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.434601 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.434672 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.434690 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 04:58:52.434788 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.432436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:43319 (local address 127.3.174.129:45227)
0808 04:58:52.432590 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.432601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.432662 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.432803 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.432804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.432806 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:52.432848 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.432849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.433467 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.433469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.434051 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.434054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.434375 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.434377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.434439 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.434454 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.434608 (+ 154us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.434610 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.434663 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.434709 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260808 04:58:52.437052 17794 raft_consensus.cc:1261] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Refusing update from remote peer 87459be0cab84bca91f13272cb87f60c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:52.437186 17526 raft_consensus.cc:1261] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Refusing update from remote peer 87459be0cab84bca91f13272cb87f60c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:52.437526 17948 consensus_queue.cc:1035] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:52.437690 17948 consensus_queue.cc:1035] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:52.446524 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.444242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:42747 (local address 127.3.174.129:46767)
0808 04:58:52.444282 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.444284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.444339 (+ 55us) server_negotiation.cc:407] Connection header received
0808 04:58:52.444449 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.444450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.444461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.444490 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:52.444638 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.445079 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.445876 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.446074 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.446133 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.446287 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.446321 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.446372 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.446438 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.446457 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 04:58:52.446553 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.444118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:46767 (local address 127.3.174.130:42747)
0808 04:58:52.444270 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.444282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.444367 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.444542 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.444543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.444544 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:52.444578 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.444579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.445158 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.445159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.445785 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.445788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.446140 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.446141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.446198 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.446215 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.446380 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.446381 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.446434 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.446479 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260808 04:58:52.455233 16979 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:52.455349 16979 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:52.456003 16979 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082: Sending DeleteTablet for 3 replicas of tablet 14bf2b9393394e298b4c816901e27f6b
I20260808 04:58:52.456285 17236 tablet_service.cc:1514] Processing DeleteTablet for tablet 14bf2b9393394e298b4c816901e27f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:57762
I20260808 04:58:52.456329 17758 tablet_service.cc:1514] Processing DeleteTablet for tablet 14bf2b9393394e298b4c816901e27f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:51986
I20260808 04:58:52.456701 17484 tablet_service.cc:1514] Processing DeleteTablet for tablet 14bf2b9393394e298b4c816901e27f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:33024
I20260808 04:58:52.456924 18052 tablet_replica.cc:330] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: stopping tablet replica
I20260808 04:58:52.456984 18052 raft_consensus.cc:2229] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:52.457028 18052 raft_consensus.cc:2258] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:52.457149 18051 tablet_replica.cc:330] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: stopping tablet replica
I20260808 04:58:52.457217 18051 raft_consensus.cc:2229] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:52.457258 18051 raft_consensus.cc:2258] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:52.457535 18052 ts_tablet_manager.cc:1905] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:52.460420 18051 ts_tablet_manager.cc:1905] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:52.461773 18051 ts_tablet_manager.cc:1918] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:52.461833 18051 log.cc:1196] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/14bf2b9393394e298b4c816901e27f6b
I20260808 04:58:52.462028 18051 ts_tablet_manager.cc:1939] T 14bf2b9393394e298b4c816901e27f6b P db428020657e4f7ba0a2fe5b8edb0629: Deleting consensus metadata
I20260808 04:58:52.462343 18052 ts_tablet_manager.cc:1918] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:52.462370 16958 catalog_manager.cc:4841] TS db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531): tablet 14bf2b9393394e298b4c816901e27f6b (table test-table [id=640cff6becd345d1a94d6aaa9dd95b2b]) successfully deleted
I20260808 04:58:52.462405 18052 log.cc:1196] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/14bf2b9393394e298b4c816901e27f6b
I20260808 04:58:52.462611 18052 ts_tablet_manager.cc:1939] T 14bf2b9393394e298b4c816901e27f6b P b7419c18963e4dd39ae31733d1264781: Deleting consensus metadata
I20260808 04:58:52.462922 16954 catalog_manager.cc:4841] TS b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319): tablet 14bf2b9393394e298b4c816901e27f6b (table test-table [id=640cff6becd345d1a94d6aaa9dd95b2b]) successfully deleted
I20260808 04:58:52.465384 18049 tablet_replica.cc:330] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: stopping tablet replica
I20260808 04:58:52.465463 18049 raft_consensus.cc:2229] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:52.465564 18049 raft_consensus.cc:2258] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:52.465817 18049 ts_tablet_manager.cc:1905] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:52.467119 18049 ts_tablet_manager.cc:1918] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:52.467172 18049 log.cc:1196] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/14bf2b9393394e298b4c816901e27f6b
I20260808 04:58:52.467386 18049 ts_tablet_manager.cc:1939] T 14bf2b9393394e298b4c816901e27f6b P 87459be0cab84bca91f13272cb87f60c: Deleting consensus metadata
I20260808 04:58:52.467703 16958 catalog_manager.cc:4841] TS 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767): tablet 14bf2b9393394e298b4c816901e27f6b (table test-table [id=640cff6becd345d1a94d6aaa9dd95b2b]) successfully deleted
Aug 08 04:58:52 dist-test-slave-gjw7 krb5kdc[16909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 04:58:52 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:52 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 04:58:52.496073 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:52.489241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.3.174.190:35363)
0808 04:58:52.489337 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.489340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.489614 (+ 274us) server_negotiation.cc:407] Connection header received
0808 04:58:52.489657 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.489659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.489675 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.489734 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:52.490432 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.491286 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.492393 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.492728 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.492799 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.496000 (+ 3201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39934: BlockingRecv error: recv got EOF from 127.0.0.1:39934 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 04:58:52 dist-test-slave-gjw7 krb5kdc[16909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM
I20260808 04:58:52.504052 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.496876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.3.174.190:35363)
0808 04:58:52.496929 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.496931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.497047 (+ 116us) server_negotiation.cc:407] Connection header received
0808 04:58:52.497102 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.497103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.497111 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.497154 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:52.497725 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.498169 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.499225 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.499404 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.499457 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.501833 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:52.501835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:52.501835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:52.501843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:52.503067 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:52.503375 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:52.503376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:52.503376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:52.503432 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:52.503593 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:52.503593 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:52.503594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:52.503698 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:52.503792 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.503888 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.503961 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 04:58:52.505604 16979 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:52.505792 16979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:52.507279 17236 tablet_service.cc:1467] Processing CreateTablet for tablet 37a39e227b1445268b7d2982dd20e845 (DEFAULT_TABLE table=test-table [id=86ce0323cf6e4226b3e62a7016e71dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:52.507287 17484 tablet_service.cc:1467] Processing CreateTablet for tablet 37a39e227b1445268b7d2982dd20e845 (DEFAULT_TABLE table=test-table [id=86ce0323cf6e4226b3e62a7016e71dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:52.507424 17236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a39e227b1445268b7d2982dd20e845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:52.507426 17484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a39e227b1445268b7d2982dd20e845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:52.508442 17937 tablet_bootstrap.cc:492] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: Bootstrap starting.
I20260808 04:58:52.508836 17937 tablet_bootstrap.cc:654] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:52.508859 17758 tablet_service.cc:1467] Processing CreateTablet for tablet 37a39e227b1445268b7d2982dd20e845 (DEFAULT_TABLE table=test-table [id=86ce0323cf6e4226b3e62a7016e71dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:52.508998 17758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a39e227b1445268b7d2982dd20e845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:52.509457 17937 tablet_bootstrap.cc:492] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: No bootstrap required, opened a new log
I20260808 04:58:52.509500 17937 ts_tablet_manager.cc:1397] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:52.509640 17937 raft_consensus.cc:348] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.509701 17937 raft_consensus.cc:374] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:52.509719 17937 raft_consensus.cc:729] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87459be0cab84bca91f13272cb87f60c, State: Initialized, Role: FOLLOWER
I20260808 04:58:52.509758 17937 consensus_queue.cc:260] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.509857 17937 ts_tablet_manager.cc:1428] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:52.510277 17939 tablet_bootstrap.cc:492] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: Bootstrap starting.
I20260808 04:58:52.510582 17939 tablet_bootstrap.cc:654] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:52.510813 17938 tablet_bootstrap.cc:492] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: Bootstrap starting.
I20260808 04:58:52.511205 17938 tablet_bootstrap.cc:654] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:52.511819 17939 tablet_bootstrap.cc:492] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: No bootstrap required, opened a new log
I20260808 04:58:52.511870 17939 ts_tablet_manager.cc:1397] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 04:58:52.511996 17939 raft_consensus.cc:348] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.512065 17939 raft_consensus.cc:374] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:52.512084 17939 raft_consensus.cc:729] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7419c18963e4dd39ae31733d1264781, State: Initialized, Role: FOLLOWER
I20260808 04:58:52.512131 17939 consensus_queue.cc:260] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.512218 17939 ts_tablet_manager.cc:1428] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:52.512437 17938 tablet_bootstrap.cc:492] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: No bootstrap required, opened a new log
I20260808 04:58:52.512488 17938 ts_tablet_manager.cc:1397] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 04:58:52.512616 17938 raft_consensus.cc:348] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.512684 17938 raft_consensus.cc:374] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:52.512702 17938 raft_consensus.cc:729] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db428020657e4f7ba0a2fe5b8edb0629, State: Initialized, Role: FOLLOWER
I20260808 04:58:52.512742 17938 consensus_queue.cc:260] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.512820 17938 ts_tablet_manager.cc:1428] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:52.646551 17990 raft_consensus.cc:482] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:52.646723 17990 raft_consensus.cc:504] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.646982 17990 leader_election.cc:288] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319), db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531)
I20260808 04:58:52.647234 17526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a39e227b1445268b7d2982dd20e845" candidate_uuid: "87459be0cab84bca91f13272cb87f60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7419c18963e4dd39ae31733d1264781" is_pre_election: true
I20260808 04:58:52.647292 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a39e227b1445268b7d2982dd20e845" candidate_uuid: "87459be0cab84bca91f13272cb87f60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db428020657e4f7ba0a2fe5b8edb0629" is_pre_election: true
I20260808 04:58:52.647356 17526 raft_consensus.cc:2454] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87459be0cab84bca91f13272cb87f60c in term 0.
I20260808 04:58:52.647372 17794 raft_consensus.cc:2454] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87459be0cab84bca91f13272cb87f60c in term 0.
I20260808 04:58:52.647516 17156 leader_election.cc:302] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87459be0cab84bca91f13272cb87f60c, b7419c18963e4dd39ae31733d1264781; no voters:
I20260808 04:58:52.647610 17990 raft_consensus.cc:2781] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:52.647650 17990 raft_consensus.cc:482] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:52.647667 17990 raft_consensus.cc:3037] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.648355 17990 raft_consensus.cc:504] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.648481 17990 leader_election.cc:288] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [CANDIDATE]: Term 1 election: Requested vote from peers b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319), db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531)
I20260808 04:58:52.648633 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a39e227b1445268b7d2982dd20e845" candidate_uuid: "87459be0cab84bca91f13272cb87f60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db428020657e4f7ba0a2fe5b8edb0629"
I20260808 04:58:52.648710 17794 raft_consensus.cc:3037] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.648711 17526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a39e227b1445268b7d2982dd20e845" candidate_uuid: "87459be0cab84bca91f13272cb87f60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7419c18963e4dd39ae31733d1264781"
I20260808 04:58:52.648818 17526 raft_consensus.cc:3037] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.649320 17794 raft_consensus.cc:2454] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87459be0cab84bca91f13272cb87f60c in term 1.
I20260808 04:58:52.649483 17159 leader_election.cc:302] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87459be0cab84bca91f13272cb87f60c, db428020657e4f7ba0a2fe5b8edb0629; no voters:
I20260808 04:58:52.649564 17990 raft_consensus.cc:2781] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:52.649603 17990 raft_consensus.cc:686] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 1 LEADER]: Becoming Leader. State: Replica: 87459be0cab84bca91f13272cb87f60c, State: Running, Role: LEADER
I20260808 04:58:52.649652 17990 consensus_queue.cc:237] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.649770 17526 raft_consensus.cc:2454] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87459be0cab84bca91f13272cb87f60c in term 1.
I20260808 04:58:52.650221 16976 catalog_manager.cc:5495] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c reported cstate change: term changed from 0 to 1, leader changed from <none> to 87459be0cab84bca91f13272cb87f60c (127.3.174.129). New cstate: current_term: 1 leader_uuid: "87459be0cab84bca91f13272cb87f60c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } health_report { overall_health: UNKNOWN } } }
I20260808 04:58:52.656283 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.653628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57788 (local address 127.3.174.129:46767)
0808 04:58:52.653688 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.653690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.653750 (+ 60us) server_negotiation.cc:407] Connection header received
0808 04:58:52.653840 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.653840 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.653849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.653898 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:52.654040 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.654513 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.655574 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.655775 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.655828 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.656031 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.656069 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.656123 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.656198 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.656218 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 04:58:52.656951 17794 raft_consensus.cc:1261] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Refusing update from remote peer 87459be0cab84bca91f13272cb87f60c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:52.656983 17526 raft_consensus.cc:1261] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Refusing update from remote peer 87459be0cab84bca91f13272cb87f60c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:52.657162 17990 consensus_queue.cc:1035] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:52.657347 17948 consensus_queue.cc:1035] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:52.659652 16979 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:52.659750 16979 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:52.660466 16979 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082: Sending DeleteTablet for 3 replicas of tablet 37a39e227b1445268b7d2982dd20e845
I20260808 04:58:52.660712 17236 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a39e227b1445268b7d2982dd20e845 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:57762
I20260808 04:58:52.660723 17234 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a39e227b1445268b7d2982dd20e845 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:57762
I20260808 04:58:52.660742 17758 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a39e227b1445268b7d2982dd20e845 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:51986
I20260808 04:58:52.660817 18049 tablet_replica.cc:330] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: stopping tablet replica
I20260808 04:58:52.660838 17484 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a39e227b1445268b7d2982dd20e845 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:33024
I20260808 04:58:52.660848 17757 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a39e227b1445268b7d2982dd20e845 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:51986
I20260808 04:58:52.660866 18049 raft_consensus.cc:2229] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:52.660910 18051 tablet_replica.cc:330] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: stopping tablet replica
I20260808 04:58:52.660923 18052 tablet_replica.cc:330] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: stopping tablet replica
I20260808 04:58:52.660923 18049 raft_consensus.cc:2258] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:52.660955 18051 raft_consensus.cc:2229] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:52.660974 18052 raft_consensus.cc:2229] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:52.661000 18051 raft_consensus.cc:2258] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:52.661012 18052 raft_consensus.cc:2258] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:52.661281 18049 ts_tablet_manager.cc:1905] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:52.661340 18052 ts_tablet_manager.cc:1905] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:52.661609 18051 ts_tablet_manager.cc:1905] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:52.662933 18051 ts_tablet_manager.cc:1918] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:52.662941 18049 ts_tablet_manager.cc:1918] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:52.662988 18051 log.cc:1196] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/37a39e227b1445268b7d2982dd20e845
I20260808 04:58:52.662999 18049 log.cc:1196] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/37a39e227b1445268b7d2982dd20e845
I20260808 04:58:52.663022 18052 ts_tablet_manager.cc:1918] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:58:52.663066 18052 log.cc:1196] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/37a39e227b1445268b7d2982dd20e845
I20260808 04:58:52.663193 18051 ts_tablet_manager.cc:1939] T 37a39e227b1445268b7d2982dd20e845 P db428020657e4f7ba0a2fe5b8edb0629: Deleting consensus metadata
I20260808 04:58:52.663240 18049 ts_tablet_manager.cc:1939] T 37a39e227b1445268b7d2982dd20e845 P 87459be0cab84bca91f13272cb87f60c: Deleting consensus metadata
I20260808 04:58:52.663247 18052 ts_tablet_manager.cc:1939] T 37a39e227b1445268b7d2982dd20e845 P b7419c18963e4dd39ae31733d1264781: Deleting consensus metadata
I20260808 04:58:52.663479 16979 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:58:52.663620 16979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:58:52.663852 16958 catalog_manager.cc:4841] TS db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531): tablet 37a39e227b1445268b7d2982dd20e845 (table test-table [id=86ce0323cf6e4226b3e62a7016e71dea]) successfully deleted
W20260808 04:58:52.664011 16958 catalog_manager.cc:4808] TS db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531): delete failed for tablet 37a39e227b1445268b7d2982dd20e845 because the tablet was not found. No further retry: Not found: Tablet not found: 37a39e227b1445268b7d2982dd20e845
I20260808 04:58:52.664098 16958 catalog_manager.cc:4841] TS 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767): tablet 37a39e227b1445268b7d2982dd20e845 (table test-table [id=86ce0323cf6e4226b3e62a7016e71dea]) successfully deleted
I20260808 04:58:52.664137 16954 catalog_manager.cc:4841] TS b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319): tablet 37a39e227b1445268b7d2982dd20e845 (table test-table [id=86ce0323cf6e4226b3e62a7016e71dea]) successfully deleted
W20260808 04:58:52.664156 16958 catalog_manager.cc:4808] TS 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767): delete failed for tablet 37a39e227b1445268b7d2982dd20e845 because the tablet was not found. No further retry: Not found: Tablet not found: 37a39e227b1445268b7d2982dd20e845
I20260808 04:58:52.665331 17757 tablet_service.cc:1467] Processing CreateTablet for tablet 1ceb0b54a1084850816ff30bbbb91607 (DEFAULT_TABLE table=test-table [id=a3dda0d25a09422fbe3251a4fbae528f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:52.665354 17484 tablet_service.cc:1467] Processing CreateTablet for tablet 1ceb0b54a1084850816ff30bbbb91607 (DEFAULT_TABLE table=test-table [id=a3dda0d25a09422fbe3251a4fbae528f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:52.665373 17234 tablet_service.cc:1467] Processing CreateTablet for tablet 1ceb0b54a1084850816ff30bbbb91607 (DEFAULT_TABLE table=test-table [id=a3dda0d25a09422fbe3251a4fbae528f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:58:52.665442 17757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ceb0b54a1084850816ff30bbbb91607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:52.665462 17484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ceb0b54a1084850816ff30bbbb91607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:52.665460 17234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ceb0b54a1084850816ff30bbbb91607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:52.666687 17937 tablet_bootstrap.cc:492] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c: Bootstrap starting.
I20260808 04:58:52.666802 17938 tablet_bootstrap.cc:492] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: Bootstrap starting.
I20260808 04:58:52.666960 17937 tablet_bootstrap.cc:654] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:52.667430 17937 tablet_bootstrap.cc:492] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c: No bootstrap required, opened a new log
I20260808 04:58:52.667449 17938 tablet_bootstrap.cc:654] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:52.667470 17937 ts_tablet_manager.cc:1397] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 04:58:52.667569 17937 raft_consensus.cc:348] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.667626 17937 raft_consensus.cc:374] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:52.667646 17937 raft_consensus.cc:729] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87459be0cab84bca91f13272cb87f60c, State: Initialized, Role: FOLLOWER
I20260808 04:58:52.667687 17937 consensus_queue.cc:260] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.667780 17937 ts_tablet_manager.cc:1428] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:52.667800 17939 tablet_bootstrap.cc:492] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: Bootstrap starting.
I20260808 04:58:52.668154 17939 tablet_bootstrap.cc:654] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:52.668758 17939 tablet_bootstrap.cc:492] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: No bootstrap required, opened a new log
I20260808 04:58:52.668808 17939 ts_tablet_manager.cc:1397] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:52.668908 17939 raft_consensus.cc:348] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.668969 17939 raft_consensus.cc:374] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:52.668994 17939 raft_consensus.cc:729] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7419c18963e4dd39ae31733d1264781, State: Initialized, Role: FOLLOWER
I20260808 04:58:52.669039 17939 consensus_queue.cc:260] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.669133 17939 ts_tablet_manager.cc:1428] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:52.669447 17938 tablet_bootstrap.cc:492] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: No bootstrap required, opened a new log
I20260808 04:58:52.669494 17938 ts_tablet_manager.cc:1397] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 04:58:52.669631 17938 raft_consensus.cc:348] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.669713 17938 raft_consensus.cc:374] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:52.669732 17938 raft_consensus.cc:729] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db428020657e4f7ba0a2fe5b8edb0629, State: Initialized, Role: FOLLOWER
I20260808 04:58:52.669780 17938 consensus_queue.cc:260] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.669862 17938 ts_tablet_manager.cc:1428] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:52.749248 17952 raft_consensus.cc:482] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:58:52.749369 17952 raft_consensus.cc:504] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.749739 17952 leader_election.cc:288] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767), b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319)
I20260808 04:58:52.752718 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.749914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:46767 (local address 127.3.174.131:60277)
0808 04:58:52.750064 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.750074 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.750136 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.750292 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.750293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.750296 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:52.750364 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.750368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.751025 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.751026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.751900 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.751903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.752485 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.752485 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.752546 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.752550 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.752594 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.752632 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260808 04:58:52.752707 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.749988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:60277 (local address 127.3.174.129:46767)
0808 04:58:52.750035 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.750037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.750136 (+ 99us) server_negotiation.cc:407] Connection header received
0808 04:58:52.750193 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.750194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.750204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.750251 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:52.750430 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.750928 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.751988 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.752414 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.752480 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.752490 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.752606 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.752640 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 04:58:52.752853 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.749992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:33105 (local address 127.3.174.130:43319)
0808 04:58:52.750046 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.750048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.750132 (+ 84us) server_negotiation.cc:407] Connection header received
0808 04:58:52.750256 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.750257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.750269 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.750335 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:58:52.750481 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.751002 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.752082 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.752538 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.752610 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.752619 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.752756 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.752791 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 04:58:52.752862 18108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.749911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:43319 (local address 127.3.174.131:33105)
0808 04:58:52.750061 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.750072 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.750138 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.750342 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.750343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.750349 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:58:52.750416 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.750419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.751107 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.751108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.751993 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.751997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.752616 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.752617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.752681 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.752685 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.752732 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.752769 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":24,"thread_start_us":71,"threads_started":1}
I20260808 04:58:52.752988 17267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ceb0b54a1084850816ff30bbbb91607" candidate_uuid: "db428020657e4f7ba0a2fe5b8edb0629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87459be0cab84bca91f13272cb87f60c" is_pre_election: true
I20260808 04:58:52.753079 17267 raft_consensus.cc:2454] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db428020657e4f7ba0a2fe5b8edb0629 in term 0.
I20260808 04:58:52.753273 17663 leader_election.cc:302] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87459be0cab84bca91f13272cb87f60c, db428020657e4f7ba0a2fe5b8edb0629; no voters:
I20260808 04:58:52.753335 17526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ceb0b54a1084850816ff30bbbb91607" candidate_uuid: "db428020657e4f7ba0a2fe5b8edb0629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7419c18963e4dd39ae31733d1264781" is_pre_election: true
I20260808 04:58:52.753439 17526 raft_consensus.cc:2454] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db428020657e4f7ba0a2fe5b8edb0629 in term 0.
I20260808 04:58:52.753439 17952 raft_consensus.cc:2781] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:58:52.753473 17952 raft_consensus.cc:482] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:58:52.753499 17952 raft_consensus.cc:3037] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.754189 17952 raft_consensus.cc:504] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.754314 17952 leader_election.cc:288] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [CANDIDATE]: Term 1 election: Requested vote from peers 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767), b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319)
I20260808 04:58:52.754465 17267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ceb0b54a1084850816ff30bbbb91607" candidate_uuid: "db428020657e4f7ba0a2fe5b8edb0629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87459be0cab84bca91f13272cb87f60c"
I20260808 04:58:52.754537 17267 raft_consensus.cc:3037] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.755084 17267 raft_consensus.cc:2454] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db428020657e4f7ba0a2fe5b8edb0629 in term 1.
I20260808 04:58:52.755183 17526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ceb0b54a1084850816ff30bbbb91607" candidate_uuid: "db428020657e4f7ba0a2fe5b8edb0629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7419c18963e4dd39ae31733d1264781"
I20260808 04:58:52.755261 17526 raft_consensus.cc:3037] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:52.755241 17663 leader_election.cc:302] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87459be0cab84bca91f13272cb87f60c, db428020657e4f7ba0a2fe5b8edb0629; no voters:
I20260808 04:58:52.755339 17952 raft_consensus.cc:2781] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:52.755502 17952 raft_consensus.cc:686] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 LEADER]: Becoming Leader. State: Replica: db428020657e4f7ba0a2fe5b8edb0629, State: Running, Role: LEADER
I20260808 04:58:52.755591 17952 consensus_queue.cc:237] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } }
I20260808 04:58:52.755970 17526 raft_consensus.cc:2454] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db428020657e4f7ba0a2fe5b8edb0629 in term 1.
I20260808 04:58:52.756358 16979 catalog_manager.cc:5495] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 reported cstate change: term changed from 0 to 1, leader changed from <none> to db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131). New cstate: current_term: 1 leader_uuid: "db428020657e4f7ba0a2fe5b8edb0629" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db428020657e4f7ba0a2fe5b8edb0629" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 43531 } health_report { overall_health: HEALTHY } } }
I20260808 04:58:52.764544 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.762219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51988 (local address 127.3.174.131:43531)
0808 04:58:52.762364 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.762366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.762373 (+ 7us) server_negotiation.cc:407] Connection header received
0808 04:58:52.762409 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.762410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.762428 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.762497 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:52.762657 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.763138 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.763840 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.764029 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.764070 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.764211 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.764316 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.764370 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.764459 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.764476 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260808 04:58:52.768959 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.766824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:35099 (local address 127.3.174.130:43319)
0808 04:58:52.766868 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.766870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.766993 (+ 123us) server_negotiation.cc:407] Connection header received
0808 04:58:52.767030 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.767031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.767039 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.767079 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:52.767171 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.767607 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.768422 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.768595 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.768636 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.768804 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.768826 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.768857 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.768897 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.768912 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 04:58:52.768988 18120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.766764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:43319 (local address 127.3.174.131:35099)
0808 04:58:52.766935 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.766943 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.766995 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.767083 (+ 88us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.767083 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.767084 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:52.767134 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.767135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.767692 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.767693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.768346 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.768349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.768667 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.768668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.768727 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.768740 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.768865 (+ 125us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.768866 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.768894 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.768926 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260808 04:58:52.770833 17526 raft_consensus.cc:1261] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Refusing update from remote peer db428020657e4f7ba0a2fe5b8edb0629: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:52.770843 17267 raft_consensus.cc:1261] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Refusing update from remote peer db428020657e4f7ba0a2fe5b8edb0629: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:58:52.771260 18111 consensus_queue.cc:1035] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7419c18963e4dd39ae31733d1264781" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:52.771386 17952 consensus_queue.cc:1035] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87459be0cab84bca91f13272cb87f60c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:58:52.777591 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.775665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:32987 (local address 127.3.174.131:43531)
0808 04:58:52.775706 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:52.775707 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:52.775713 (+ 6us) server_negotiation.cc:407] Connection header received
0808 04:58:52.775730 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:52.775731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:52.775739 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:52.775776 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:58:52.775868 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.776338 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.776984 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.777147 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.777229 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.777385 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.777418 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.777461 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:52.777513 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 04:58:52.777531 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 04:58:52.777617 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.775585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:43531 (local address 127.3.174.130:32987)
0808 04:58:52.775618 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.775628 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:52.775668 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:52.775780 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:52.775780 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:52.775781 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 04:58:52.775819 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.775821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.776402 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.776402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.776925 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:52.776926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:52.777230 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:52.777231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:52.777305 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:52.777312 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 04:58:52.777466 (+ 154us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 04:58:52.777470 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:58:52.777511 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 04:58:52.777557 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 04:58:52.784348 16979 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:52.784436 16979 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:58:52.785022 16979 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bceacd09f5ac4db7b25adfab5afc7082: Sending DeleteTablet for 3 replicas of tablet 1ceb0b54a1084850816ff30bbbb91607
I20260808 04:58:52.785275 17757 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ceb0b54a1084850816ff30bbbb91607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:51986
I20260808 04:58:52.785275 17234 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ceb0b54a1084850816ff30bbbb91607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:57762
I20260808 04:58:52.785400 18049 tablet_replica.cc:330] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c: stopping tablet replica
I20260808 04:58:52.785459 18049 raft_consensus.cc:2229] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:52.785498 18049 raft_consensus.cc:2258] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:52.785640 18051 tablet_replica.cc:330] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: stopping tablet replica
I20260808 04:58:52.785712 18051 raft_consensus.cc:2229] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:58:52.785781 18049 ts_tablet_manager.cc:1905] T 1ceb0b54a1084850816ff30bbbb91607 P 87459be0cab84bca91f13272cb87f60c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:52.785800 18051 raft_consensus.cc:2258] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:52.786312 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 17124
I20260808 04:58:52.803292 17484 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ceb0b54a1084850816ff30bbbb91607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:58:52 UTC) from {username='oryx', principal='oryx/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:33024
I20260808 04:58:52.803428 18052 tablet_replica.cc:330] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: stopping tablet replica
I20260808 04:58:52.803481 18052 raft_consensus.cc:2229] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:58:52.803519 18052 raft_consensus.cc:2258] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:58:52.804720 18052 ts_tablet_manager.cc:1905] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:52.806792 18052 ts_tablet_manager.cc:1918] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:52.806862 18052 log.cc:1196] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/1ceb0b54a1084850816ff30bbbb91607
I20260808 04:58:52.806895 18051 ts_tablet_manager.cc:1905] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:58:52.807055 18052 ts_tablet_manager.cc:1939] T 1ceb0b54a1084850816ff30bbbb91607 P b7419c18963e4dd39ae31733d1264781: Deleting consensus metadata
I20260808 04:58:52.808197 16954 catalog_manager.cc:4841] TS b7419c18963e4dd39ae31733d1264781 (127.3.174.130:43319): tablet 1ceb0b54a1084850816ff30bbbb91607 (table test-table [id=a3dda0d25a09422fbe3251a4fbae528f]) successfully deleted
I20260808 04:58:52.808300 18051 ts_tablet_manager.cc:1918] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 04:58:52.808373 18051 log.cc:1196] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/1ceb0b54a1084850816ff30bbbb91607
I20260808 04:58:52.808586 18051 ts_tablet_manager.cc:1939] T 1ceb0b54a1084850816ff30bbbb91607 P db428020657e4f7ba0a2fe5b8edb0629: Deleting consensus metadata
I20260808 04:58:52.808933 16958 catalog_manager.cc:4841] TS db428020657e4f7ba0a2fe5b8edb0629 (127.3.174.131:43531): tablet 1ceb0b54a1084850816ff30bbbb91607 (table test-table [id=a3dda0d25a09422fbe3251a4fbae528f]) successfully deleted
W20260808 04:58:52.813663 16958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:46767 (error 108)
I20260808 04:58:52.813871 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 17365
I20260808 04:58:52.819706 18131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:52.819469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:46767 (local address 127.0.0.1:57804)
0808 04:58:52.819603 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:52.819648 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:46767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":49,"threads_started":1}
W20260808 04:58:52.819809 16958 catalog_manager.cc:4584] TS 87459be0cab84bca91f13272cb87f60c (127.3.174.129:46767): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ceb0b54a1084850816ff30bbbb91607: Network error: Client connection negotiation failed: client connection to 127.3.174.129:46767: connect: Connection refused (error 111)
I20260808 04:58:52.830078 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 17631
I20260808 04:58:52.840454 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 16930
2026-08-08T04:58:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3137 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:52 dist-test-slave-gjw7 krb5kdc[18137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:52 dist-test-slave-gjw7 krb5kdc[18137](info): set up 2 sockets
Aug 08 04:58:52 dist-test-slave-gjw7 krb5kdc[18137](info): commencing operation
krb5kdc: starting...
W20260808 04:58:54.027083 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:54 dist-test-slave-gjw7 krb5kdc[18137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:54.152076 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:43497
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:54.240685 18157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:54.240922 18157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:54.240945 18157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:54.242116 18157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:54.242168 18157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:54.242184 18157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:54.242197 18157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:54.242209 18157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:54.243737 18157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:43497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:54.243964 18157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:54.244169 18157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:54.249282 18166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:54.249333 18170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:54.249315 18157 server_base.cc:1034] running on GCE node
W20260808 04:58:54.249330 18167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:54.249738 18157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:54.250036 18157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:54.251232 18157 hybrid_clock.cc:648] HybridClock initialized: now 1786165134251205 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T04:58:54Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1412 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:58:54 dist-test-slave-gjw7 krb5kdc[18177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:58:54 dist-test-slave-gjw7 krb5kdc[18177](info): set up 2 sockets
Aug 08 04:58:54 dist-test-slave-gjw7 krb5kdc[18177](info): commencing operation
krb5kdc: starting...
W20260808 04:58:55.450559 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:58:55 dist-test-slave-gjw7 krb5kdc[18177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:58:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:58:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:58:55.569147 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35877
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:35877
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:55.667549 18203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:55.667738 18203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:55.667758 18203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:55.668804 18203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:55.668838 18203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:55.668850 18203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:55.668869 18203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:55.668886 18203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:55.670387 18203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:35877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:55.670588 18203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:55.670753 18203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:55.675308 18216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:55.675350 18220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:55.675350 18214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:55.675498 18203 server_base.cc:1034] running on GCE node
I20260808 04:58:55.675746 18203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:55.676028 18203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:55.677181 18203 hybrid_clock.cc:648] HybridClock initialized: now 1786165135677164 us; error 36 us; skew 500 ppm
Aug 08 04:58:55 dist-test-slave-gjw7 krb5kdc[18177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165135, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:55.679629 18203 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:55.680058 18203 webserver.cc:460] Webserver started at http://127.3.174.190:36969/ using document root <none> and password file <none>
I20260808 04:58:55.680289 18203 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:55.680338 18203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:55.680483 18203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:55.681731 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "c3cf3639f09b4c74bc87a288ef5e3a63"
format_stamp: "Formatted at 2026-08-08 04:58:55 on dist-test-slave-gjw7"
server_key: "002c6780d6670172df40fdd5fae83eea"
server_key_iv: "ba869c78f648c5722aaf822257406b6d"
server_key_version: "encryptionkey@0"
I20260808 04:58:55.682068 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "c3cf3639f09b4c74bc87a288ef5e3a63"
format_stamp: "Formatted at 2026-08-08 04:58:55 on dist-test-slave-gjw7"
server_key: "002c6780d6670172df40fdd5fae83eea"
server_key_iv: "ba869c78f648c5722aaf822257406b6d"
server_key_version: "encryptionkey@0"
I20260808 04:58:55.683331 18203 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:58:55.683990 18229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:55.684157 18203 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:55.684265 18203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "c3cf3639f09b4c74bc87a288ef5e3a63"
format_stamp: "Formatted at 2026-08-08 04:58:55 on dist-test-slave-gjw7"
server_key: "002c6780d6670172df40fdd5fae83eea"
server_key_iv: "ba869c78f648c5722aaf822257406b6d"
server_key_version: "encryptionkey@0"
I20260808 04:58:55.684330 18203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:55.699940 18203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:55.701771 18203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:55.701903 18203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:55.706568 18203 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:35877
I20260808 04:58:55.706601 18328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:35877 every 8 connection(s)
I20260808 04:58:55.706938 18203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:55.707839 18330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:58:55.710160 18330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: Bootstrap starting.
I20260808 04:58:55.710724 18330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: Neither blocks nor log segments found. Creating new log.
I20260808 04:58:55.711001 18330 log.cc:824] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:55.711686 18330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: No bootstrap required, opened a new log
I20260808 04:58:55.713349 18330 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } }
I20260808 04:58:55.713508 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 18203
I20260808 04:58:55.713531 18330 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:58:55.713564 18330 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3cf3639f09b4c74bc87a288ef5e3a63, State: Initialized, Role: FOLLOWER
I20260808 04:58:55.713613 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:58:55.713711 18330 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } }
I20260808 04:58:55.713794 18330 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:55.713821 3770 external_mini_cluster.cc:1428] Setting key 2a064daafc4d2b58f56ad7ffd0c214c0
I20260808 04:58:55.713840 18330 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:55.713881 18330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:58:55.714471 18330 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } }
I20260808 04:58:55.714584 18330 leader_election.cc:302] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3cf3639f09b4c74bc87a288ef5e3a63; no voters:
I20260808 04:58:55.714740 18330 leader_election.cc:288] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:58:55.714778 18336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:58:55.714985 18336 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 1 LEADER]: Becoming Leader. State: Replica: c3cf3639f09b4c74bc87a288ef5e3a63, State: Running, Role: LEADER
I20260808 04:58:55.714988 18330 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:55.715147 18336 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } }
I20260808 04:58:55.715518 18338 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } } }
I20260808 04:58:55.715595 18338 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:55.715829 18342 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3cf3639f09b4c74bc87a288ef5e3a63. Latest consensus state: current_term: 1 leader_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } } }
I20260808 04:58:55.715969 18342 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:55.716150 18348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:55.716691 18348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:55.718415 18348 catalog_manager.cc:1347] Generated new cluster ID: 4653001408ed43b2ae7a727db75af9c1
I20260808 04:58:55.718478 18348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 04:58:55 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:55.725236 18334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:55.714559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.3.174.190:35877)
0808 04:58:55.714709 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:55.714720 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:55.714759 (+ 39us) server_negotiation.cc:407] Connection header received
0808 04:58:55.714878 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:55.714886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:55.714948 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:55.715129 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:55.715979 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:55.717314 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:55.718242 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:55.718782 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:55.718880 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:55.721765 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:55.721770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:55.721770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:55.721782 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:55.723641 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:55.724036 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:55.724038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:55.724039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:55.724151 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:55.724358 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:55.724358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:55.724358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:55.724628 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:55.724766 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:55.724889 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 04:58:55.725017 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260808 04:58:55.734218 18348 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:58:55.734916 18348 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:55.740026 18348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: Generated new TSK 0
I20260808 04:58:55.740200 18348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:58:55.767773 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35877
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:55.850107 18377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:55.850299 18377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:55.850322 18377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:55.851450 18377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:55.851488 18377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:55.851516 18377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:55.852985 18377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:55.853216 18377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:55.853461 18377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:55.858331 18384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:55.858362 18387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:55.858336 18383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:55.858481 18377 server_base.cc:1034] running on GCE node
I20260808 04:58:55.858754 18377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:55.858999 18377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:55.860136 18377 hybrid_clock.cc:648] HybridClock initialized: now 1786165135860124 us; error 27 us; skew 500 ppm
Aug 08 04:58:55 dist-test-slave-gjw7 krb5kdc[18177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165135, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:55.862648 18377 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:58:55.863042 18377 webserver.cc:460] Webserver started at http://127.3.174.129:40619/ using document root <none> and password file <none>
I20260808 04:58:55.863276 18377 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:55.863324 18377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:55.863447 18377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:55.864562 18377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "8426c908d0a34e2d94c3654c9d1a502f"
format_stamp: "Formatted at 2026-08-08 04:58:55 on dist-test-slave-gjw7"
server_key: "e7ffa504c3e165327149dc89967f34f5"
server_key_iv: "1a0d80067ebd6db0b51b617181f70f78"
server_key_version: "encryptionkey@0"
I20260808 04:58:55.864897 18377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "8426c908d0a34e2d94c3654c9d1a502f"
format_stamp: "Formatted at 2026-08-08 04:58:55 on dist-test-slave-gjw7"
server_key: "e7ffa504c3e165327149dc89967f34f5"
server_key_iv: "1a0d80067ebd6db0b51b617181f70f78"
server_key_version: "encryptionkey@0"
I20260808 04:58:55.866140 18377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:58:55.866827 18401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:55.866976 18377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:58:55.867058 18377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "8426c908d0a34e2d94c3654c9d1a502f"
format_stamp: "Formatted at 2026-08-08 04:58:55 on dist-test-slave-gjw7"
server_key: "e7ffa504c3e165327149dc89967f34f5"
server_key_iv: "1a0d80067ebd6db0b51b617181f70f78"
server_key_version: "encryptionkey@0"
I20260808 04:58:55.867120 18377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:55.891670 18377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:55.893610 18377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:55.893739 18377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:55.893960 18377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:55.894256 18377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:55.894291 18377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:55.894320 18377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:55.894343 18377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:55.899871 18377 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:38317
I20260808 04:58:55.899931 18603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:38317 every 8 connection(s)
I20260808 04:58:55.900208 18377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:58:55.902666 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 18377
I20260808 04:58:55.902797 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:58:55.902968 3770 external_mini_cluster.cc:1428] Setting key cdd58f2ee9cb4f185b63f6a3bc551edf
Aug 08 04:58:55 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165135, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:55.909585 18334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:55.900872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:52183 (local address 127.3.174.190:35877)
0808 04:58:55.900924 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:55.900926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:55.901131 (+ 205us) server_negotiation.cc:407] Connection header received
0808 04:58:55.901664 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:55.901665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:55.901679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:55.901735 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:55.902652 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:55.903286 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:55.904226 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:55.904440 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:55.904494 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:55.906694 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:55.906695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:55.906695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:55.906703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:55.908122 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:55.908604 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:55.908606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:55.908606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:55.908676 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:55.908906 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:55.908907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:55.908907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:55.909043 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:55.909111 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:55.909400 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 04:58:55.909492 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 04:58:55.909649 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:55.900852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:52183)
0808 04:58:55.901012 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:55.901055 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:55.901566 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:55.901788 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:55.901791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:55.902345 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:55.902570 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:55.902577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:55.903368 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:55.903369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:55.904145 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:55.904147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:55.904512 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:55.904513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:55.904634 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:55.905089 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:55.905097 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:55.906605 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:55.908267 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:55.908275 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:55.908278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:55.908515 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:55.908753 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:55.908754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:55.908754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:55.908852 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:55.909100 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:55.909103 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:55.909252 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 04:58:55.909365 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 04:58:55.909489 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260808 04:58:55.910387 18604 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35877
I20260808 04:58:55.910478 18604 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:55.910650 18604 heartbeater.cc:511] Master 127.3.174.190:35877 requested a full tablet report, sending...
I20260808 04:58:55.911170 18269 ts_manager.cc:194] Registered new tserver with Master: 8426c908d0a34e2d94c3654c9d1a502f (127.3.174.129:38317)
I20260808 04:58:55.911770 18269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:52183
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:58:55.929530 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35877
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:56.010524 18612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:56.010707 18612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:56.010730 18612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:56.011821 18612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:56.011854 18612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:56.011878 18612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:56.013262 18612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:56.013537 18612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:56.013760 18612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:56.018710 18620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.018798 18621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:56.018913 18612 server_base.cc:1034] running on GCE node
W20260808 04:58:56.018711 18624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:56.019130 18612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:56.019383 18612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:56.020529 18612 hybrid_clock.cc:648] HybridClock initialized: now 1786165136020513 us; error 28 us; skew 500 ppm
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165136, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:56.022964 18612 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:58:56.023464 18612 webserver.cc:460] Webserver started at http://127.3.174.130:39559/ using document root <none> and password file <none>
I20260808 04:58:56.023693 18612 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:56.023746 18612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:56.023880 18612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:56.024911 18612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "3ef4b2e8556d43f490f62864973e9770"
format_stamp: "Formatted at 2026-08-08 04:58:56 on dist-test-slave-gjw7"
server_key: "c02f695ad9a03da5091dacdb7ee64af1"
server_key_iv: "de0e9769e681bf05fefde6e10ff58586"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.025274 18612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "3ef4b2e8556d43f490f62864973e9770"
format_stamp: "Formatted at 2026-08-08 04:58:56 on dist-test-slave-gjw7"
server_key: "c02f695ad9a03da5091dacdb7ee64af1"
server_key_iv: "de0e9769e681bf05fefde6e10ff58586"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.026631 18612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:56.027361 18635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.027539 18612 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:56.027635 18612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "3ef4b2e8556d43f490f62864973e9770"
format_stamp: "Formatted at 2026-08-08 04:58:56 on dist-test-slave-gjw7"
server_key: "c02f695ad9a03da5091dacdb7ee64af1"
server_key_iv: "de0e9769e681bf05fefde6e10ff58586"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.027699 18612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:56.044831 18612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:56.046840 18612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:56.046985 18612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:56.047233 18612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:56.047539 18612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:56.047572 18612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.047600 18612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:56.047617 18612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.053320 18612 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:44839
I20260808 04:58:56.053377 18838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:44839 every 8 connection(s)
I20260808 04:58:56.053674 18612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:58:56.054181 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 18612
I20260808 04:58:56.054262 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:58:56.054461 3770 external_mini_cluster.cc:1428] Setting key ea054370f38a178f233786f154cc60db
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165136, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:56.063536 18334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.054421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:39847 (local address 127.3.174.190:35877)
0808 04:58:56.054482 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.054485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.054737 (+ 252us) server_negotiation.cc:407] Connection header received
0808 04:58:56.055318 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.055319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.055330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.055382 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.056342 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.056959 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.058067 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.058338 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.058409 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.060697 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:56.060698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:56.060699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:56.060707 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:56.062225 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:56.062589 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:56.062590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:56.062591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:56.062660 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:56.062888 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:56.062889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:56.062889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:56.063040 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:56.063111 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:56.063349 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 04:58:56.063443 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
I20260808 04:58:56.063680 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.054421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:39847)
0808 04:58:56.054603 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.054645 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.055218 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.055457 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.055461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.056050 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.056245 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.056251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.057059 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.057060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.057976 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.057980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.058417 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.058418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.058530 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.059015 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.059024 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.060602 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:56.062300 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:56.062306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:56.062308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:56.062518 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:56.062720 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:56.062721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:56.062721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:56.062830 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:56.063097 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:56.063100 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:56.063211 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 04:58:56.063304 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 04:58:56.063466 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260808 04:58:56.064180 18839 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35877
I20260808 04:58:56.064281 18839 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:56.064455 18839 heartbeater.cc:511] Master 127.3.174.190:35877 requested a full tablet report, sending...
I20260808 04:58:56.064811 18269 ts_manager.cc:194] Registered new tserver with Master: 3ef4b2e8556d43f490f62864973e9770 (127.3.174.130:44839)
I20260808 04:58:56.065227 18269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:39847
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:58:56.080000 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35877
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:56.161945 18846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:56.162138 18846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:56.162168 18846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:56.163323 18846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:56.163360 18846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:56.163393 18846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:56.164825 18846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:56.165056 18846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:56.165266 18846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:56.169975 18862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.170151 18856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.170336 18852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:56.170373 18846 server_base.cc:1034] running on GCE node
I20260808 04:58:56.170572 18846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:56.170794 18846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:56.172019 18846 hybrid_clock.cc:648] HybridClock initialized: now 1786165136172003 us; error 31 us; skew 500 ppm
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165136, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:56.174491 18846 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:58:56.174937 18846 webserver.cc:460] Webserver started at http://127.3.174.131:36525/ using document root <none> and password file <none>
I20260808 04:58:56.175158 18846 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:56.175213 18846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:56.175340 18846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:58:56.176424 18846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "7811edc9801d4c13bfdb4b528513b851"
format_stamp: "Formatted at 2026-08-08 04:58:56 on dist-test-slave-gjw7"
server_key: "771cc16a51f10549f4f4635e23578802"
server_key_iv: "9e796f785755ead90d1a4fecd2935d0c"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.176752 18846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "7811edc9801d4c13bfdb4b528513b851"
format_stamp: "Formatted at 2026-08-08 04:58:56 on dist-test-slave-gjw7"
server_key: "771cc16a51f10549f4f4635e23578802"
server_key_iv: "9e796f785755ead90d1a4fecd2935d0c"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.178061 18846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:58:56.178752 18873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.178901 18846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:56.178984 18846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "7811edc9801d4c13bfdb4b528513b851"
format_stamp: "Formatted at 2026-08-08 04:58:56 on dist-test-slave-gjw7"
server_key: "771cc16a51f10549f4f4635e23578802"
server_key_iv: "9e796f785755ead90d1a4fecd2935d0c"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.179044 18846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:56.199733 18846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:56.201593 18846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:56.201733 18846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:56.201951 18846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:56.202253 18846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:56.202288 18846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.202316 18846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:56.202342 18846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.207880 18846 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:45433
I20260808 04:58:56.207919 19077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:45433 every 8 connection(s)
I20260808 04:58:56.208223 18846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165136, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:58:56.215025 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 18846
I20260808 04:58:56.215137 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:58:56.215349 3770 external_mini_cluster.cc:1428] Setting key 5d36eb407bdb2f63dede4974097da228
I20260808 04:58:56.217729 18334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.208929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:42977 (local address 127.3.174.190:35877)
0808 04:58:56.208989 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.208991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.209164 (+ 173us) server_negotiation.cc:407] Connection header received
0808 04:58:56.209702 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.209703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.209713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.209766 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.210620 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.211100 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.212106 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.212325 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.212382 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.214560 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:56.214562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:56.214562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:56.214570 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:56.216156 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:56.216677 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:56.216678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:56.216679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:56.216754 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:56.216995 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:56.216996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:56.216996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:56.217115 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:56.217173 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:56.217538 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 04:58:56.217619 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:58:56.217859 19081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.208886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:42977)
0808 04:58:56.209039 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.209079 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.209619 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.209822 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.209825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.210364 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.210556 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.210560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.211172 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.211172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.212025 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.212028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.212399 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.212400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.212512 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.213056 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.213067 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.214458 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:58:56.216232 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:56.216238 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:56.216241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:56.216585 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:56.216812 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:58:56.216812 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:58:56.216813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:58:56.216924 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:58:56.217172 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:58:56.217180 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:58:56.217411 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 04:58:56.217506 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 04:58:56.217651 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260808 04:58:56.218253 19078 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35877
I20260808 04:58:56.218346 19078 heartbeater.cc:464] Registering TS with master...
I20260808 04:58:56.218518 19078 heartbeater.cc:511] Master 127.3.174.190:35877 requested a full tablet report, sending...
I20260808 04:58:56.218835 18269 ts_manager.cc:194] Registered new tserver with Master: 7811edc9801d4c13bfdb4b528513b851 (127.3.174.131:45433)
I20260808 04:58:56.219169 18269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:42977
I20260808 04:58:56.227020 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab.
Entry for principal oryx/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab.
WARNING: no policy specified for oryx/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab.
Entry for principal oryx/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab.
WARNING: no policy specified for oryx/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab.
Entry for principal oryx/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab.
I20260808 04:58:56.299535 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 18377
I20260808 04:58:56.312237 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 18612
I20260808 04:58:56.327955 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 18846
I20260808 04:58:56.334316 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 18203
I20260808 04:58:56.343868 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35877
--webserver_interface=127.3.174.190
--webserver_port=36969
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:35877
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:58:56.428886 19091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:56.429096 19091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:56.429118 19091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:56.430299 19091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:58:56.430348 19091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:56.430364 19091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:56.430378 19091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:58:56.430405 19091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:58:56.431918 19091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:35877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=36969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:56.432165 19091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:56.432384 19091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:56.437342 19102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.437265 19103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.437395 19108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:56.437543 19091 server_base.cc:1034] running on GCE node
I20260808 04:58:56.437775 19091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:56.438017 19091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:56.439169 19091 hybrid_clock.cc:648] HybridClock initialized: now 1786165136439154 us; error 33 us; skew 500 ppm
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165136, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:56.441779 19091 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:58:56.442174 19091 webserver.cc:460] Webserver started at http://127.3.174.190:36969/ using document root <none> and password file <none>
I20260808 04:58:56.442354 19091 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:56.442407 19091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:56.443681 19091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:56.444321 19117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.444556 19091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 04:58:56.444674 19091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "c3cf3639f09b4c74bc87a288ef5e3a63"
format_stamp: "Formatted at 2026-08-08 04:58:55 on dist-test-slave-gjw7"
server_key: "002c6780d6670172df40fdd5fae83eea"
server_key_iv: "ba869c78f648c5722aaf822257406b6d"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.445158 19091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:56.463032 19091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:56.465047 19091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:56.465153 19091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:56.469000 19091 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:35877
I20260808 04:58:56.469038 19204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:35877 every 8 connection(s)
I20260808 04:58:56.469441 19091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:58:56.470955 19205 sys_catalog.cc:263] Verifying existing consensus state
I20260808 04:58:56.471685 19205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: Bootstrap starting.
I20260808 04:58:56.479139 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 19091
I20260808 04:58:56.479470 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:38317
--local_ip_for_outbound_sockets=127.3.174.129
--tserver_master_addrs=127.3.174.190:35877
--webserver_port=40619
--webserver_interface=127.3.174.129
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.174.129
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 04:58:56.480211 19205 log.cc:824] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: Log is configured to *not* fsync() on all Append() calls
I20260808 04:58:56.482059 19205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 04:58:56.482331 19205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: Bootstrap complete.
I20260808 04:58:56.484522 19205 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } }
I20260808 04:58:56.484759 19205 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3cf3639f09b4c74bc87a288ef5e3a63, State: Initialized, Role: FOLLOWER
I20260808 04:58:56.484922 19205 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } }
I20260808 04:58:56.484999 19205 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:58:56.485049 19205 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:58:56.485096 19205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 1 FOLLOWER]: Advancing to term 2
I20260808 04:58:56.486160 19205 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } }
I20260808 04:58:56.486271 19205 leader_election.cc:302] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3cf3639f09b4c74bc87a288ef5e3a63; no voters:
I20260808 04:58:56.486433 19205 leader_election.cc:288] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 04:58:56.486482 19210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 04:58:56.486627 19210 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [term 2 LEADER]: Becoming Leader. State: Replica: c3cf3639f09b4c74bc87a288ef5e3a63, State: Running, Role: LEADER
I20260808 04:58:56.486711 19205 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:58:56.486758 19210 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } }
I20260808 04:58:56.487100 19212 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3cf3639f09b4c74bc87a288ef5e3a63. Latest consensus state: current_term: 2 leader_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } } }
I20260808 04:58:56.487112 19211 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cf3639f09b4c74bc87a288ef5e3a63" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35877 } } }
I20260808 04:58:56.487219 19212 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:56.487226 19211 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63 [sys.catalog]: This master's current role is: LEADER
I20260808 04:58:56.487926 19218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:58:56.488638 19218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:58:56.488796 19218 catalog_manager.cc:1259] Loaded cluster ID: 4653001408ed43b2ae7a727db75af9c1
I20260808 04:58:56.488838 19218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:58:56.490041 19218 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:58:56.490218 19218 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c3cf3639f09b4c74bc87a288ef5e3a63: Loaded TSK: 0
I20260808 04:58:56.490447 19218 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 04:58:56.564306 19209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:56.564492 19209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:56.564513 19209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:56.565735 19209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:56.565778 19209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:56.565802 19209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:58:56.567250 19209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.129.keytab
--principal=oryx/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:38317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=40619
--tserver_master_addrs=127.3.174.190:35877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:56.567456 19209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:56.567667 19209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:56.572425 19256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.572487 19247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.572427 19251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:56.573484 19209 server_base.cc:1034] running on GCE node
I20260808 04:58:56.573690 19209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:56.573916 19209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:56.575052 19209 hybrid_clock.cc:648] HybridClock initialized: now 1786165136575039 us; error 35 us; skew 500 ppm
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165136, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:56.577463 19209 init.cc:377] Logged in from keytab as oryx/127.3.174.129@KRBTEST.COM (short username oryx)
I20260808 04:58:56.577903 19209 webserver.cc:460] Webserver started at http://127.3.174.129:40619/ using document root <none> and password file <none>
I20260808 04:58:56.578135 19209 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:56.578202 19209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:56.579582 19209 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:56.580271 19276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.580438 19209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:56.580528 19209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "8426c908d0a34e2d94c3654c9d1a502f"
format_stamp: "Formatted at 2026-08-08 04:58:55 on dist-test-slave-gjw7"
server_key: "e7ffa504c3e165327149dc89967f34f5"
server_key_iv: "1a0d80067ebd6db0b51b617181f70f78"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.580884 19209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:56.590391 19209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:56.592423 19209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:56.592535 19209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:56.592751 19209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:56.593037 19209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:56.593068 19209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.593089 19209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:56.593112 19209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.599067 19209 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:38317
I20260808 04:58:56.599125 19475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:38317 every 8 connection(s)
I20260808 04:58:56.599439 19209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:58:56.605376 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 19209
I20260808 04:58:56.605669 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:44839
--local_ip_for_outbound_sockets=127.3.174.130
--tserver_master_addrs=127.3.174.190:35877
--webserver_port=39559
--webserver_interface=127.3.174.130
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.174.130
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:56.608482 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.600063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:37579)
0808 04:58:56.600235 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.600283 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.601021 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.601502 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.601507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.602195 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.602448 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.602454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.603544 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.603545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.604754 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.604756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.605359 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.605361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.605488 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.605976 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.605985 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.608293 (+ 2308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
W20260808 04:58:56.608719 19476 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:35877 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.174.190:35877: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
W20260808 04:58:56.608871 19480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:56.600176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:37579 (local address 127.3.174.190:35877)
0808 04:58:56.600355 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.600364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.600397 (+ 33us) server_negotiation.cc:407] Connection header received
0808 04:58:56.601229 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.601235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.601312 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.601432 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.602559 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.603449 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.604837 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.605237 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.605371 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.608442 (+ 3071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:37579: BlockingRecv error: recv got EOF from 127.3.174.129:37579 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:56.614346 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.608932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:47293)
0808 04:58:56.609053 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.609063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.609110 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.609590 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.609591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.609719 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.609770 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.609772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.610609 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.610610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.611364 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.611367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.611816 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.611817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.611879 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.612292 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.612297 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.614276 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
W20260808 04:58:56.614441 19480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:56.608974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:47293 (local address 127.3.174.190:35877)
0808 04:58:56.609011 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.609013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.609452 (+ 439us) server_negotiation.cc:407] Connection header received
0808 04:58:56.609481 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.609482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.609494 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.609553 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.609842 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.610520 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.611446 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.611745 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.611806 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.614383 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:47293: BlockingRecv error: recv got EOF from 127.3.174.129:47293 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:56.619462 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.614864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:50459)
0808 04:58:56.614960 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.614971 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.615020 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.615148 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.615149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.615288 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.615324 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.615325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.616142 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.616143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.616774 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.616776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.617210 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.617211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.617265 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.617681 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.617686 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.619394 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 04:58:56.619544 19480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:56.614920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:50459 (local address 127.3.174.190:35877)
0808 04:58:56.614962 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.614965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.615007 (+ 42us) server_negotiation.cc:407] Connection header received
0808 04:58:56.615077 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.615078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.615089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.615143 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.615392 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.616063 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.616854 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.617131 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.617200 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.619486 (+ 2286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:50459: BlockingRecv error: recv got EOF from 127.3.174.129:50459 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 04:58:56.619624 19476 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:58:56.690641 19482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:56.690840 19482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:56.690871 19482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:56.691992 19482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:56.692030 19482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:56.692065 19482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:58:56.693490 19482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.130.keytab
--principal=oryx/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:44839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=39559
--tserver_master_addrs=127.3.174.190:35877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:56.693715 19482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:56.693922 19482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:56.698665 19493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.698642 19496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.698658 19491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:56.698686 19482 server_base.cc:1034] running on GCE node
I20260808 04:58:56.699013 19482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:56.699232 19482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:56.700433 19482 hybrid_clock.cc:648] HybridClock initialized: now 1786165136700416 us; error 29 us; skew 500 ppm
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165136, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:56.702766 19482 init.cc:377] Logged in from keytab as oryx/127.3.174.130@KRBTEST.COM (short username oryx)
I20260808 04:58:56.703138 19482 webserver.cc:460] Webserver started at http://127.3.174.130:39559/ using document root <none> and password file <none>
I20260808 04:58:56.703312 19482 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:56.703364 19482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:56.704638 19482 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:56.705331 19512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.705493 19482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:58:56.705576 19482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "3ef4b2e8556d43f490f62864973e9770"
format_stamp: "Formatted at 2026-08-08 04:58:56 on dist-test-slave-gjw7"
server_key: "c02f695ad9a03da5091dacdb7ee64af1"
server_key_iv: "de0e9769e681bf05fefde6e10ff58586"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.705874 19482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:56.713811 19482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:56.715822 19482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:56.715972 19482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:56.716146 19482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:56.716401 19482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:56.716431 19482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.716449 19482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:56.716466 19482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.721854 19482 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:44839
I20260808 04:58:56.721920 19704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:44839 every 8 connection(s)
I20260808 04:58:56.722206 19482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 04:58:56.728868 19480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:56.722840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:50077 (local address 127.3.174.190:35877)
0808 04:58:56.722918 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.722920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.723067 (+ 147us) server_negotiation.cc:407] Connection header received
0808 04:58:56.723593 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.723594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.723610 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.723660 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.724542 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.725041 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.726046 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.726234 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.726290 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.728814 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:50077: BlockingRecv error: recv got EOF from 127.3.174.130:50077 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 04:58:56.728896 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.722806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:50077)
0808 04:58:56.722957 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.722995 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.723491 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.723727 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.723731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.724266 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.724444 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.724449 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.725123 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.725124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.725961 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.725964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.726303 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.726303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.726422 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.726793 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.726800 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.728715 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
W20260808 04:58:56.729091 19707 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:35877 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.174.190:35877: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:56.732236 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 19482
I20260808 04:58:56.732494 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:45433
--local_ip_for_outbound_sockets=127.3.174.131
--tserver_master_addrs=127.3.174.190:35877
--webserver_port=36525
--webserver_interface=127.3.174.131
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.174.131
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:56.733134 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.729310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:50851)
0808 04:58:56.729366 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.729376 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.729422 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.729556 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.729556 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.729660 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.729704 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.729705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.730248 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.730248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.730765 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.730767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.731032 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.731033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.731080 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.731467 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.731472 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.733069 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 04:58:56.733196 19480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:56.729419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:50851 (local address 127.3.174.190:35877)
0808 04:58:56.729466 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.729468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.729476 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:56.729498 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.729499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.729505 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.729544 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.729752 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.730175 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.730823 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.730981 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.731018 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.733151 (+ 2133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:50851: BlockingRecv error: recv got EOF from 127.3.174.130:50851 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:56.737519 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.733376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:41191)
0808 04:58:56.733489 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.733498 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.733531 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.733654 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.733654 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.733753 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.733776 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.733777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.734440 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.734440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.735106 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.735109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.735564 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.735564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.735601 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.735964 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.735967 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.737478 (+ 1511us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
W20260808 04:58:56.737612 19707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 04:58:56.737639 19480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:56.733472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:41191 (local address 127.3.174.190:35877)
0808 04:58:56.733507 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.733509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.733520 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:58:56.733592 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.733593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.733601 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.733650 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.733828 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.734369 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.735250 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.735481 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.735563 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.737583 (+ 2020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:41191: BlockingRecv error: recv got EOF from 127.3.174.130:41191 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 04:58:56.816129 19712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:58:56.816319 19712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:58:56.816342 19712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:58:56.817591 19712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:58:56.817631 19712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:58:56.817654 19712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:58:56.819067 19712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/krb5kdc/oryx_127.3.174.131.keytab
--principal=oryx/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:45433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=36525
--tserver_master_addrs=127.3.174.190:35877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:58:56.819295 19712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:58:56.819509 19712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:58:56.824337 19723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.824352 19722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:58:56.824368 19726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:58:56.824551 19712 server_base.cc:1034] running on GCE node
I20260808 04:58:56.824811 19712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:58:56.825071 19712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:58:56.826236 19712 hybrid_clock.cc:648] HybridClock initialized: now 1786165136826216 us; error 31 us; skew 500 ppm
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165136, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:58:56.828778 19712 init.cc:377] Logged in from keytab as oryx/127.3.174.131@KRBTEST.COM (short username oryx)
I20260808 04:58:56.829183 19712 webserver.cc:460] Webserver started at http://127.3.174.131:36525/ using document root <none> and password file <none>
I20260808 04:58:56.829437 19712 fs_manager.cc:359] Metadata directory not provided
I20260808 04:58:56.829502 19712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:58:56.830914 19712 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 04:58:56.831496 19741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.831664 19712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:58:56.831743 19712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "7811edc9801d4c13bfdb4b528513b851"
format_stamp: "Formatted at 2026-08-08 04:58:56 on dist-test-slave-gjw7"
server_key: "771cc16a51f10549f4f4635e23578802"
server_key_iv: "9e796f785755ead90d1a4fecd2935d0c"
server_key_version: "encryptionkey@0"
I20260808 04:58:56.832082 19712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:58:56.842427 19712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:58:56.844184 19712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:58:56.844306 19712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:58:56.844511 19712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:58:56.844807 19712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:58:56.844841 19712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.844861 19712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:58:56.844883 19712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:58:56.850557 19712 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:45433
I20260808 04:58:56.850621 19928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:45433 every 8 connection(s)
I20260808 04:58:56.850934 19712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:56.858073 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.851617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:41587)
0808 04:58:56.851770 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.851807 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.852329 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.852578 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.852582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.853127 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.853372 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.853379 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.854039 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.854040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.854844 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.854846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.855209 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.855210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.855337 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.855789 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.855798 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.857937 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
W20260808 04:58:56.858089 19480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:56.851637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:41587 (local address 127.3.174.190:35877)
0808 04:58:56.851700 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.851703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.851885 (+ 182us) server_negotiation.cc:407] Connection header received
0808 04:58:56.852440 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.852441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.852455 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.852506 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.853466 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.853956 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.854933 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.855131 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.855192 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.858040 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:41587: BlockingRecv error: recv got EOF from 127.3.174.131:41587 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 04:58:56.858338 19929 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:35877 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.174.190:35877: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
I20260808 04:58:56.858493 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 19712
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:56.863327 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.858558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:39143)
0808 04:58:56.858741 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.858752 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.858793 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.858933 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.858934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.859072 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.859127 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.859128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.859822 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.859824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.860635 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.860638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.861078 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.861079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.861134 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.861534 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.861540 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.863264 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167}
W20260808 04:58:56.863415 19480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:56.858738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:39143 (local address 127.3.174.190:35877)
0808 04:58:56.858801 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.858803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.858816 (+ 13us) server_negotiation.cc:407] Connection header received
0808 04:58:56.858845 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.858846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.858857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.858930 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.859194 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.859736 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.860721 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.860989 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.861067 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.863358 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:39143: BlockingRecv error: recv got EOF from 127.3.174.131:39143 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 04:58:56.866387 19934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.858857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38662 (local address 127.3.174.190:35877)
0808 04:58:56.859049 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.859051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.859084 (+ 33us) server_negotiation.cc:407] Connection header received
0808 04:58:56.859153 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.859222 (+ 69us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.859238 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.859314 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.859897 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.860630 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.861561 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.861865 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.861941 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.863158 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:58:56.863163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:58:56.863163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:58:56.863174 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:58:56.864919 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:56.865319 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:56.865323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:56.865323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:56.865422 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:58:56.865628 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:58:56.865628 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:58:56.865628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:58:56.865912 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:58:56.866085 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 04:58:56.866195 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 04:58:56.866305 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":61,"threads_started":1}
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:56 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:56.868345 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:56.863784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:33491)
0808 04:58:56.863826 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:56.863836 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:56.863880 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:56.864227 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:56.864228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:56.864337 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:56.864363 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.864364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.865140 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.865142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.865771 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:56.865774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.866216 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.866217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:56.866271 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.866771 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:56.866776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:56.868291 (+ 1515us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 04:58:56.868405 19480 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:56.863827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:33491 (local address 127.3.174.190:35877)
0808 04:58:56.864089 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:56.864091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:56.864099 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:58:56.864118 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:56.864119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:56.864128 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:56.864195 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:56.864426 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.865073 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.865842 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:56.866097 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:56.866155 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:56.868362 (+ 2207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:33491: BlockingRecv error: recv got EOF from 127.3.174.131:33491 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 04:58:56.868471 19929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 04:58:57 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:57 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:57.624660 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:57.620021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:47373)
0808 04:58:57.620159 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:57.620170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:57.620228 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:57.620390 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:57.620391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:57.620558 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:57.620609 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:57.620611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.621374 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.621375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:57.621965 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:57.621967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.622360 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.622360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:57.622402 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:57.622821 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:57.622825 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:57.624592 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
W20260808 04:58:57.624788 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:57.620132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:47373 (local address 127.3.174.190:35877)
0808 04:58:57.620265 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:57.620267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:57.620276 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:57.620298 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:57.620299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:57.620309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:57.620373 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:57.620672 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.621245 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.622048 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.622291 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.622355 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:57.624713 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:47373: BlockingRecv error: recv got EOF from 127.3.174.129:47373 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
Aug 08 04:58:57 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:57 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:57.742524 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:57.737974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:45609)
0808 04:58:57.738094 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:57.738106 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:57.738163 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:57.738325 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:57.738326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:57.738492 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:57.738541 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:57.738543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.739218 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.739220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:57.739794 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:57.739796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.740166 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.740167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:57.740206 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:57.740635 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:57.740640 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:57.742453 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
W20260808 04:58:57.742632 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:57.738043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:45609 (local address 127.3.174.190:35877)
0808 04:58:57.738108 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:57.738109 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:57.738149 (+ 40us) server_negotiation.cc:407] Connection header received
0808 04:58:57.738253 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:57.738254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:57.738264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:57.738304 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:57.738618 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.739113 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.739892 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.740102 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.740161 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:57.742569 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:45609: BlockingRecv error: recv got EOF from 127.3.174.130:45609 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 04:58:57 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:57 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:57.873543 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:57.868862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:50923)
0808 04:58:57.869003 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:57.869017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:57.869094 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:57.869284 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:57.869315 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:57.869498 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:57.869545 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:57.869547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.870193 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.870194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:57.870763 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:57.870766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.871112 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.871113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:57.871169 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:57.871648 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:57.871653 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:57.873440 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260808 04:58:57.873598 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:57.868953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:50923 (local address 127.3.174.190:35877)
0808 04:58:57.869034 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:57.869036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:57.869084 (+ 48us) server_negotiation.cc:407] Connection header received
0808 04:58:57.869155 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:57.869156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:57.869170 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:57.869268 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:57.869620 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.870120 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.870852 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:57.871045 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:57.871105 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:57.873542 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:50923: BlockingRecv error: recv got EOF from 127.3.174.131:50923 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 04:58:58 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:58 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:58.633875 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:58.625148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:36055)
0808 04:58:58.625356 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:58.625368 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:58.625430 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:58.625603 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:58.625605 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:58.625792 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:58.625844 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:58.625846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.626492 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.626493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:58.627022 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:58.627024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.627327 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.627327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:58.627363 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:58.632063 (+ 4700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:58.632068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:58.633816 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":116,"threads_started":1}
W20260808 04:58:58.634020 19954 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:58.625236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:36055 (local address 127.3.174.190:35877)
0808 04:58:58.625383 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:58.625385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:58.625413 (+ 28us) server_negotiation.cc:407] Connection header received
0808 04:58:58.625481 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:58.625482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:58.625493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:58.625553 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:58.625919 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.626430 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.627082 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.627276 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.627324 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:58.633937 (+ 6613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:36055: BlockingRecv error: recv got EOF from 127.3.174.129:36055 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
Aug 08 04:58:58 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:58 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:58.747522 19961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:58.743032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:56935)
0808 04:58:58.743162 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:58.743182 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:58.743258 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:58.743383 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:58.743384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:58.743552 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:58.743602 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:58.743604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.744205 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.744205 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:58.744738 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:58.744740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.745050 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.745050 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:58.745088 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:58.745549 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:58.745554 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:58.747458 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
W20260808 04:58:58.747591 19954 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:58.743134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:56935 (local address 127.3.174.190:35877)
0808 04:58:58.743205 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:58.743208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:58.743239 (+ 31us) server_negotiation.cc:407] Connection header received
0808 04:58:58.743311 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:58.743311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:58.743321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:58.743368 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:58.743651 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.744152 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.744807 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.744997 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.745048 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:58.747530 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:56935: BlockingRecv error: recv got EOF from 127.3.174.130:56935 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 04:58:58 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:58 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:58.878505 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:58.874064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:51655)
0808 04:58:58.874204 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:58.874215 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:58.874271 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:58.874398 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:58.874399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:58.874579 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:58.874628 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:58.874629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.875296 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.875297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:58.875934 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:58.875937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.876251 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.876252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:58.876306 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:58.876726 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:58.876730 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:58.878424 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
W20260808 04:58:58.878571 19954 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:58.874212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:51655 (local address 127.3.174.190:35877)
0808 04:58:58.874277 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:58.874279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:58.874290 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:58:58.874326 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:58.874327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:58.874341 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:58.874382 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:58.874702 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.875218 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.876008 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:58.876192 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:58.876245 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:58.878507 (+ 2262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:51655: BlockingRecv error: recv got EOF from 127.3.174.131:51655 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 04:58:59 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:59 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:59.639068 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:59.634355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:49085)
0808 04:58:59.634485 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:59.634496 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:59.634549 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:59.634773 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:59.634775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:59.634938 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:59.634990 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:59.634992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.635669 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.635670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:59.636240 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:59.636242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.636626 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.636627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:59.636674 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:59.637117 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:59.637121 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:59.638988 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
W20260808 04:58:59.639158 19964 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:59.634401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:49085 (local address 127.3.174.190:35877)
0808 04:58:59.634520 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:59.634522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:59.634531 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:58:59.634651 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:59.634652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:59.634665 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:59.634754 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:59.635052 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.635595 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.636310 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.636558 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.636612 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:59.639093 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:49085: BlockingRecv error: recv got EOF from 127.3.174.129:49085 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
Aug 08 04:58:59 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:59 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:59.752648 19966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:59.747947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:59847)
0808 04:58:59.748087 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:59.748100 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:59.748175 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:59.748340 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:59.748342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:59.748513 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:59.748567 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:59.748568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.749202 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.749203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:59.749779 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:59.749781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.750109 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.750111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:59.750172 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:59.750630 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:59.750634 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:59.752578 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
W20260808 04:58:59.752790 19964 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:59.748013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:59847 (local address 127.3.174.190:35877)
0808 04:58:59.748064 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:59.748066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:59.748160 (+ 94us) server_negotiation.cc:407] Connection header received
0808 04:58:59.748234 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:59.748234 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:59.748245 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:59.748297 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:59.748646 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.749135 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.749846 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.750029 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.750073 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:59.752721 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:59847: BlockingRecv error: recv got EOF from 127.3.174.130:59847 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 04:58:59 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:58:59 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:58:59.884840 19967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:58:59.878958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:46631)
0808 04:58:59.879098 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 04:58:59.879110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:58:59.879174 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:58:59.879339 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:58:59.879341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:58:59.879558 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:58:59.879612 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:59.879614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.880622 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.880623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:59.881536 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:58:59.881540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.881999 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.881999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:58:59.882051 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:59.882574 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:58:59.882580 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:58:59.884757 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
W20260808 04:58:59.884907 19964 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:58:59.879120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:46631 (local address 127.3.174.190:35877)
0808 04:58:59.879178 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 04:58:59.879180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:58:59.879232 (+ 52us) server_negotiation.cc:407] Connection header received
0808 04:58:59.879256 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:58:59.879257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:58:59.879271 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:58:59.879322 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:58:59.879710 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.880518 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.881614 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:58:59.881927 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:58:59.882006 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:58:59.884853 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:46631: BlockingRecv error: recv got EOF from 127.3.174.131:46631 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 04:59:00 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:00 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:00.643911 19968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:00.639520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:55477)
0808 04:59:00.639638 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:00.639652 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:00.639726 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:00.639847 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:00.639847 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:00.640006 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:00.640056 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:00.640057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.640669 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.640670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:00.641201 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:00.641203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.641577 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.641577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:00.641639 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:00.642061 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:00.642065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:00.643829 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
W20260808 04:59:00.644001 19970 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:00.639582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:55477 (local address 127.3.174.190:35877)
0808 04:59:00.639728 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:00.639730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:00.639756 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:59:00.639777 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:00.639777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:00.639787 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:00.639835 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:00.640107 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.640611 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.641265 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.641495 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.641546 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:00.643923 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:55477: BlockingRecv error: recv got EOF from 127.3.174.129:55477 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
Aug 08 04:59:00 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:00 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:00.757732 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:00.753244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:48067)
0808 04:59:00.753413 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:00.753425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:00.753493 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:00.753632 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:00.753633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:00.753802 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:00.753854 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:00.753855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.754578 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.754579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:00.755097 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:00.755099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.755439 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.755440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:00.755477 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:00.755937 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:00.755943 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:00.757670 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
W20260808 04:59:00.757846 19970 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:00.753362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:48067 (local address 127.3.174.190:35877)
0808 04:59:00.753417 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:00.753419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:00.753472 (+ 53us) server_negotiation.cc:407] Connection header received
0808 04:59:00.753555 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:00.753556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:00.753570 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:00.753617 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:00.753941 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.754497 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.755166 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.755386 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.755437 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:00.757783 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:48067: BlockingRecv error: recv got EOF from 127.3.174.130:48067 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 04:59:00 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:00 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:00.889952 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:00.885436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:56697)
0808 04:59:00.885597 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:00.885609 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:00.885680 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:00.885817 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:00.885818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:00.885998 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:00.886052 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:00.886054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.886700 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.886701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:00.887232 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:00.887234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.887587 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.887589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:00.887641 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:00.888141 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:00.888146 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:00.889889 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
W20260808 04:59:00.890050 19970 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:00.885509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:56697 (local address 127.3.174.190:35877)
0808 04:59:00.885566 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:00.885567 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:00.885662 (+ 95us) server_negotiation.cc:407] Connection header received
0808 04:59:00.885746 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:00.885747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:00.885759 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:00.885797 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:00.886109 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.886623 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.887305 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:00.887501 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:00.887554 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:00.889988 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:56697: BlockingRecv error: recv got EOF from 127.3.174.131:56697 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 04:59:01 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:01 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:01.649144 19974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:01.644671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:49847)
0808 04:59:01.644820 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:01.644831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:01.644870 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:01.644990 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:01.644991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:01.645152 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:01.645200 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:01.645202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.645889 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.645889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:01.646413 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:01.646415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.646742 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.646743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:01.646780 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:01.647222 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:01.647227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:01.649085 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
W20260808 04:59:01.649322 19975 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:01.644723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:49847 (local address 127.3.174.190:35877)
0808 04:59:01.644849 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:01.644851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:01.644876 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:59:01.644919 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:01.644920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:01.644933 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:01.644985 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:01.645274 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.645828 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.646474 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.646686 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.646742 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:01.649212 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:49847: BlockingRecv error: recv got EOF from 127.3.174.129:49847 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
Aug 08 04:59:01 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:01 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:01.762667 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:01.758276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:45337)
0808 04:59:01.758437 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:01.758448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:01.758497 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:01.758641 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:01.758642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:01.758818 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:01.758867 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:01.758869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.759505 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.759506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:01.760034 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:01.760035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.760343 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.760344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:01.760380 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:01.760822 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:01.760826 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:01.762606 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
W20260808 04:59:01.762825 19975 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:01.758406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:45337 (local address 127.3.174.190:35877)
0808 04:59:01.758468 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:01.758470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:01.758482 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:59:01.758555 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:01.758556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:01.758568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:01.758631 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:01.758926 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.759436 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.760096 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.760290 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.760342 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:01.762752 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:45337: BlockingRecv error: recv got EOF from 127.3.174.130:45337 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 04:59:01 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:01 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:01.895206 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:01.890436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:43819)
0808 04:59:01.890614 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:01.890626 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:01.890712 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:01.890838 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:01.890839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:01.891018 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:01.891064 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:01.891066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.891767 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.891768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:01.892462 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:01.892465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.892780 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.892780 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:01.892818 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:01.893218 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:01.893222 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:01.895099 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
W20260808 04:59:01.895226 19975 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:01.890514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:43819 (local address 127.3.174.190:35877)
0808 04:59:01.890608 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:01.890610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:01.890694 (+ 84us) server_negotiation.cc:407] Connection header received
0808 04:59:01.890766 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:01.890767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:01.890782 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:01.890827 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:01.891136 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.891675 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.892532 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:01.892723 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:01.892773 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:01.895176 (+ 2403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:43819: BlockingRecv error: recv got EOF from 127.3.174.131:43819 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 04:59:02 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:02 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:02.654078 19980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:02.649583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:50487)
0808 04:59:02.649729 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:02.649740 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:02.649796 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:02.649924 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:02.649925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:02.650107 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:02.650161 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:02.650162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.650795 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.650796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:02.651343 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:02.651345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.651674 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.651675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:02.651711 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:02.652161 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:02.652166 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:02.654017 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
W20260808 04:59:02.654241 19981 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:02.649667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:50487 (local address 127.3.174.190:35877)
0808 04:59:02.649782 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:02.649784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:02.649792 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:59:02.649851 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:02.649852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:02.649864 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:02.649912 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:02.650213 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.650727 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.651415 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.651620 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.651686 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:02.654135 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:50487: BlockingRecv error: recv got EOF from 127.3.174.129:50487 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
Aug 08 04:59:02 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:02 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:02.768098 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:02.763231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:51773)
0808 04:59:02.763377 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:02.763390 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:02.763459 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:02.763591 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:02.763592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:02.763757 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:02.763811 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:02.763813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.764466 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.764468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:02.765107 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:02.765110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.765584 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.765585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:02.765629 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:02.766079 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:02.766084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:02.768017 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
W20260808 04:59:02.768177 19981 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:02.763351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:51773 (local address 127.3.174.190:35877)
0808 04:59:02.763405 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:02.763406 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:02.763440 (+ 34us) server_negotiation.cc:407] Connection header received
0808 04:59:02.763518 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:02.763519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:02.763530 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:02.763577 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:02.763880 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.764386 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.765209 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.765509 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.765580 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:02.768123 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:51773: BlockingRecv error: recv got EOF from 127.3.174.130:51773 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 04:59:02 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:02 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:02.900516 19984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:02.895720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:53681)
0808 04:59:02.895852 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:02.895863 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:02.895936 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:02.896059 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:02.896059 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:02.896235 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:02.896286 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:02.896288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.896910 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.896911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:02.897539 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:02.897542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.897926 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.897928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:02.897998 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:02.898525 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:02.898530 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:02.900454 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
W20260808 04:59:02.900676 19981 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:02.895785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:53681 (local address 127.3.174.190:35877)
0808 04:59:02.895861 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:02.895863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:02.895917 (+ 54us) server_negotiation.cc:407] Connection header received
0808 04:59:02.895989 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:02.895989 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:02.896002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:02.896046 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:02.896341 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.896848 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.897622 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:02.897832 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:02.897887 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:02.900557 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:53681: BlockingRecv error: recv got EOF from 127.3.174.131:53681 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 04:59:03 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:03 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:03.662703 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:03.654511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:46573)
0808 04:59:03.654693 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:03.654707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:03.654766 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:03.654887 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:03.654888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:03.655052 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:03.655104 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:03.655105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.655755 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.655755 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:03.656307 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:03.656309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.656648 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.656650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:03.656719 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:03.660653 (+ 3934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:03.660657 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:03.662627 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
W20260808 04:59:03.662843 19987 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:03.654611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:46573 (local address 127.3.174.190:35877)
0808 04:59:03.654743 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:03.654745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:03.654755 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:59:03.654811 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:03.654812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:03.654824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:03.654876 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:03.655151 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.655685 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.656371 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.656564 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.656609 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:03.662756 (+ 6147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:46573: BlockingRecv error: recv got EOF from 127.3.174.129:46573 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
Aug 08 04:59:03 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:03 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:03.772965 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:03.768600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:50695)
0808 04:59:03.768729 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:03.768742 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:03.768806 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:03.768939 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:03.768940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:03.769107 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:03.769156 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:03.769158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.769844 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.769845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:03.770403 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:03.770405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.770752 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.770753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:03.770804 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:03.771236 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:03.771240 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:03.772880 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
W20260808 04:59:03.773039 19987 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:03.768704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:50695 (local address 127.3.174.190:35877)
0808 04:59:03.768780 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:03.768782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:03.768791 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:59:03.768865 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:03.768866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:03.768878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:03.768921 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:03.769222 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.769772 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.770479 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.770693 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.770749 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:03.772986 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:50695: BlockingRecv error: recv got EOF from 127.3.174.130:50695 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 04:59:03 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:03 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:03.905709 19990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:03.901008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:58065)
0808 04:59:03.901165 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:03.901187 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:03.901261 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:03.901420 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:03.901421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:03.901601 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:03.901648 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:03.901650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.902371 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.902372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:03.902884 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:03.902885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.903202 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.903203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:03.903257 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:03.903699 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:03.903704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:03.905632 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
W20260808 04:59:03.905771 19987 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:03.901094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:58065 (local address 127.3.174.190:35877)
0808 04:59:03.901169 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:03.901172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:03.901247 (+ 75us) server_negotiation.cc:407] Connection header received
0808 04:59:03.901345 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:03.901346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:03.901361 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:03.901407 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:03.901732 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.902291 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.902953 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:03.903136 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:03.903192 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:03.905717 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:58065: BlockingRecv error: recv got EOF from 127.3.174.131:58065 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 04:59:04 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:04 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:04.667564 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:04.663165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:45409)
0808 04:59:04.663332 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:04.663343 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:04.663395 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:04.663543 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:04.663544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:04.663725 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:04.663776 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:04.663777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.664445 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.664445 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:04.664969 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:04.664970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.665315 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.665317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:04.665364 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:04.665767 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:04.665771 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:04.667495 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":76,"threads_started":1}
W20260808 04:59:04.667661 19992 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:04.663306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:45409 (local address 127.3.174.190:35877)
0808 04:59:04.663420 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:04.663422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:04.663430 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:59:04.663457 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:04.663458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:04.663468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:04.663530 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:04.663874 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.664387 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.665034 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.665224 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.665273 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:04.667580 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:45409: BlockingRecv error: recv got EOF from 127.3.174.129:45409 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
Aug 08 04:59:04 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:04 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:04.778347 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:04.773547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:57959)
0808 04:59:04.773716 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:04.773729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:04.773803 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:04.773923 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:04.773924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:04.774097 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:04.774149 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:04.774151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.774935 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.774936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:04.775500 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:04.775502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.775854 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.775855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:04.775898 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:04.776363 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:04.776368 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:04.778250 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
W20260808 04:59:04.778430 19992 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:04.773617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:57959 (local address 127.3.174.190:35877)
0808 04:59:04.773694 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:04.773697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:04.773783 (+ 86us) server_negotiation.cc:407] Connection header received
0808 04:59:04.773857 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:04.773858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:04.773869 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:04.773916 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:04.774230 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.774855 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.775600 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.775799 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.775845 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:04.778364 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:57959: BlockingRecv error: recv got EOF from 127.3.174.130:57959 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 04:59:04 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:04 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:04.910619 19995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:04.906164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:44819)
0808 04:59:04.906300 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:04.906313 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:04.906382 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:04.906552 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:04.906553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:04.906730 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:04.906782 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:04.906783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.907423 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.907424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:04.907997 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:04.907999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.908348 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.908349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:04.908389 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:04.908797 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:04.908802 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:04.910548 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
W20260808 04:59:04.910701 19992 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:04.906239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:44819 (local address 127.3.174.190:35877)
0808 04:59:04.906315 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:04.906316 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:04.906362 (+ 46us) server_negotiation.cc:407] Connection header received
0808 04:59:04.906435 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:04.906436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:04.906448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:04.906494 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:04.906846 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.907358 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.908057 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:04.908269 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:04.908342 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:04.910637 (+ 2295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:44819: BlockingRecv error: recv got EOF from 127.3.174.131:44819 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 04:59:05 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:05 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:05.674204 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:05.668102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:48703)
0808 04:59:05.668426 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:05.668437 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:05.668496 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:05.668654 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:05.668655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:05.668864 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:05.668924 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:05.668926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.669937 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.669938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:05.670770 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:05.670772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.671247 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.671248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:05.671312 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:05.671755 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:05.671760 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:05.674131 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":59,"threads_started":1}
W20260808 04:59:05.674489 19999 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:05.668167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:48703 (local address 127.3.174.190:35877)
0808 04:59:05.668413 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:05.668415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:05.668497 (+ 82us) server_negotiation.cc:407] Connection header received
0808 04:59:05.668550 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:05.668551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:05.668565 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:05.668632 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:05.668998 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.669848 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.670854 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.671155 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.671244 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:05.674404 (+ 3160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:48703: BlockingRecv error: recv got EOF from 127.3.174.129:48703 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
Aug 08 04:59:05 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:05 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:05.783107 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:05.778895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:58867)
0808 04:59:05.779034 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:05.779045 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:05.779105 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:05.779216 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:05.779217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:05.779380 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:05.779430 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:05.779431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.780019 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.780020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:05.780556 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:05.780559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.780887 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.780888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:05.780923 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:05.781353 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:05.781358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:05.783047 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
W20260808 04:59:05.783314 19999 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:05.778951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:58867 (local address 127.3.174.190:35877)
0808 04:59:05.779001 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:05.779003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:05.779097 (+ 94us) server_negotiation.cc:407] Connection header received
0808 04:59:05.779146 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:05.779147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:05.779158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:05.779204 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:05.779476 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.779959 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.780640 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.780835 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.780887 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:05.783216 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:58867: BlockingRecv error: recv got EOF from 127.3.174.130:58867 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 04:59:05 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:05 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:05.915521 20004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:05.911029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:39257)
0808 04:59:05.911143 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:05.911154 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:05.911205 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:05.911325 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:05.911326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:05.911500 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:05.911551 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:05.911552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.912187 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.912188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:05.912752 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:05.912755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.913079 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.913080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:05.913121 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:05.913599 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:05.913605 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:05.915459 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
W20260808 04:59:05.915616 19999 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:05.911111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:39257 (local address 127.3.174.190:35877)
0808 04:59:05.911182 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:05.911184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:05.911196 (+ 12us) server_negotiation.cc:407] Connection header received
0808 04:59:05.911249 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:05.911250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:05.911262 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:05.911313 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:05.911599 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.912098 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.912829 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:05.913015 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:05.913068 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:05.915550 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:39257: BlockingRecv error: recv got EOF from 127.3.174.131:39257 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 04:59:06 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:06 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:06.680359 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:06.675396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:49629)
0808 04:59:06.675546 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:06.675557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:06.675614 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:06.675781 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:06.675782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:06.675965 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:06.676015 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:06.676017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.676713 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.676714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:06.677256 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:06.677259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.677740 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.677741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:06.677791 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:06.678322 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:06.678327 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:06.680287 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
W20260808 04:59:06.680485 20010 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:06.675510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:49629 (local address 127.3.174.190:35877)
0808 04:59:06.675637 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:06.675639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:06.675648 (+ 9us) server_negotiation.cc:407] Connection header received
0808 04:59:06.675690 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:06.675691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:06.675703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:06.675768 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:06.676105 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.676617 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.677397 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.677650 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.677720 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:06.680411 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:49629: BlockingRecv error: recv got EOF from 127.3.174.129:49629 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
Aug 08 04:59:06 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:06 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:06.788179 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:06.783542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:45341)
0808 04:59:06.783680 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:06.783698 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:06.783758 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:06.783877 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:06.783879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:06.784045 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:06.784098 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:06.784100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.784884 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.784885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:06.785451 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:06.785454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.785769 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.785769 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:06.785806 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:06.786287 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:06.786293 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:06.788106 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
W20260808 04:59:06.788255 20010 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:06.783646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:45341 (local address 127.3.174.190:35877)
0808 04:59:06.783696 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:06.783698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:06.783744 (+ 46us) server_negotiation.cc:407] Connection header received
0808 04:59:06.783801 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:06.783802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:06.783813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:06.783862 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:06.784192 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.784778 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.785518 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.785704 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.785770 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:06.788193 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:45341: BlockingRecv error: recv got EOF from 127.3.174.130:45341 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 04:59:06 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:06 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:06.920423 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:06.915964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:39413)
0808 04:59:06.916100 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:06.916113 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:06.916182 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:06.916304 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:06.916305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:06.916486 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:06.916533 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:06.916535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.917165 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.917166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:06.917720 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:06.917723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.918046 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.918047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:06.918109 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:06.918524 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:06.918528 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:06.920329 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
W20260808 04:59:06.920488 20010 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:06.916073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:39413 (local address 127.3.174.190:35877)
0808 04:59:06.916135 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:06.916136 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:06.916161 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:59:06.916236 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:06.916237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:06.916252 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:06.916298 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:06.916601 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.917094 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.917796 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:06.917986 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:06.918042 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:06.920433 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:39413: BlockingRecv error: recv got EOF from 127.3.174.131:39413 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 04:59:07 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:07 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:07.685513 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:07.680838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:48413)
0808 04:59:07.681021 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:07.681035 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:07.681100 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:07.681244 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:07.681245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:07.681485 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:07.681535 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:07.681537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.682158 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.682159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:07.682836 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:07.682840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.683170 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.683171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:07.683252 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:07.683638 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:07.683641 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:07.685444 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
W20260808 04:59:07.685595 20016 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:07.680924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:48413 (local address 127.3.174.190:35877)
0808 04:59:07.681045 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:07.681047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:07.681082 (+ 35us) server_negotiation.cc:407] Connection header received
0808 04:59:07.681142 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:07.681143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:07.681155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:07.681219 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:07.681598 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.682100 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.682897 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.683113 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.683158 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:07.685530 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:48413: BlockingRecv error: recv got EOF from 127.3.174.129:48413 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
Aug 08 04:59:07 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:07 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:07.793527 20017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:07.788629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:51025)
0808 04:59:07.788806 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:07.788816 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:07.788882 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:07.789197 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:07.789199 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:07.789445 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:07.789496 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:07.789498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.790188 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.790190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:07.790767 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:07.790769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.791146 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.791147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:07.791196 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:07.791632 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:07.791636 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:07.793450 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":109,"threads_started":1}
W20260808 04:59:07.793582 20016 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:07.788671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:51025 (local address 127.3.174.190:35877)
0808 04:59:07.788744 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:07.788746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:07.788862 (+ 116us) server_negotiation.cc:407] Connection header received
0808 04:59:07.789052 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:07.789053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:07.789081 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:07.789151 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:07.789565 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.790091 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.790858 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.791071 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.791136 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:07.793529 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:51025: BlockingRecv error: recv got EOF from 127.3.174.130:51025 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 04:59:07 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:07 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:07.925446 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:07.920877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:47731)
0808 04:59:07.920999 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:07.921010 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:07.921067 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:07.921214 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:07.921214 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:07.921410 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:07.921457 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:07.921459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.922061 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.922062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:07.922642 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:07.922645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.923027 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.923028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:07.923076 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:07.923561 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:07.923566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:07.925350 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
W20260808 04:59:07.925484 20016 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:07.920947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:47731 (local address 127.3.174.190:35877)
0808 04:59:07.920994 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:07.920996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:07.921050 (+ 54us) server_negotiation.cc:407] Connection header received
0808 04:59:07.921105 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:07.921105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:07.921116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:07.921165 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:07.921512 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.921998 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.922745 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:07.922949 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:07.923002 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:07.925428 (+ 2426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:47731: BlockingRecv error: recv got EOF from 127.3.174.131:47731 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 04:59:08 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:08 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:08.693467 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:08.686016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:47237)
0808 04:59:08.686158 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:08.686169 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:08.686283 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:08.686465 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:08.686466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:08.686642 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:08.686692 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:08.686694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.687431 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.687433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:08.688028 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:08.688031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.688511 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.688512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:08.688577 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:08.691628 (+ 3051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:08.691633 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:08.693405 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
W20260808 04:59:08.693573 20027 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:08.686078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:47237 (local address 127.3.174.190:35877)
0808 04:59:08.686219 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:08.686221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:08.686276 (+ 55us) server_negotiation.cc:407] Connection header received
0808 04:59:08.686374 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:08.686375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:08.686388 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:08.686451 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:08.686757 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.687286 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.688110 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.688392 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.688470 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:08.693496 (+ 5026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:47237: BlockingRecv error: recv got EOF from 127.3.174.129:47237 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
Aug 08 04:59:08 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:08 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:08.798516 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:08.793995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:56993)
0808 04:59:08.794153 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:08.794165 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:08.794230 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:08.794360 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:08.794361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:08.794537 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:08.794587 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:08.794589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.795258 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.795260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:08.795928 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:08.795932 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.796271 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.796272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:08.796316 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:08.796746 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:08.796751 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:08.798456 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
W20260808 04:59:08.798633 20027 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:08.794059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:56993 (local address 127.3.174.190:35877)
0808 04:59:08.794109 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:08.794111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:08.794218 (+ 107us) server_negotiation.cc:407] Connection header received
0808 04:59:08.794286 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:08.794286 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:08.794297 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:08.794344 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:08.794643 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.795184 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.796008 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.796211 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.796263 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:08.798563 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:56993: BlockingRecv error: recv got EOF from 127.3.174.130:56993 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 04:59:08 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:08 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:08.930610 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:08.925959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:46305)
0808 04:59:08.926118 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:08.926132 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:08.926204 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:08.926369 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:08.926371 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:08.926556 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:08.926602 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:08.926604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.927236 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.927238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:08.927785 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:08.927787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.928108 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.928109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:08.928164 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:08.928630 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:08.928635 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:08.930533 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
W20260808 04:59:08.930667 20027 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:08.926079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:46305 (local address 127.3.174.190:35877)
0808 04:59:08.926158 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:08.926161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:08.926187 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:59:08.926292 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:08.926293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:08.926306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:08.926351 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:08.926675 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.927168 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.927866 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:08.928050 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:08.928105 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:08.930619 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:46305: BlockingRecv error: recv got EOF from 127.3.174.131:46305 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 04:59:09 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:09 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:09.698810 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:09.693920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:53199)
0808 04:59:09.694056 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:09.694067 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:09.694115 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:09.694261 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:09.694262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:09.694438 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:09.694488 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:09.694490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.695166 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.695167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:09.695781 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:09.695785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.696147 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.696147 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:09.696186 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:09.696696 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:09.696702 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:09.698714 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
W20260808 04:59:09.698876 20034 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:09.693996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:53199 (local address 127.3.174.190:35877)
0808 04:59:09.694123 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:09.694125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:09.694133 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:59:09.694156 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:09.694157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:09.694171 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:09.694248 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:09.694547 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.695078 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.695867 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.696083 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.696145 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:09.698813 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:53199: BlockingRecv error: recv got EOF from 127.3.174.129:53199 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
Aug 08 04:59:09 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:09 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:09.803244 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:09.798962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:46989)
0808 04:59:09.799086 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:09.799099 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:09.799151 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:09.799275 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:09.799276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:09.799444 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:09.799491 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:09.799493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.800095 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.800096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:09.800639 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:09.800641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.800951 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.800952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:09.800992 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:09.801453 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:09.801457 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:09.803141 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
W20260808 04:59:09.803294 20034 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:09.799072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:46989 (local address 127.3.174.190:35877)
0808 04:59:09.799129 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:09.799130 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:09.799140 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:59:09.799208 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:09.799209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:09.799222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:09.799268 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:09.799540 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.800039 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.800719 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.800900 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.800946 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:09.803240 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:46989: BlockingRecv error: recv got EOF from 127.3.174.130:46989 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 04:59:09 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:09 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:09.936030 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:09.931099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:36417)
0808 04:59:09.931236 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:09.931245 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:09.931304 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:09.931425 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:09.931426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:09.931605 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:09.931663 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:09.931665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.932338 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.932340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:09.933079 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:09.933096 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.933559 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.933560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:09.933605 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:09.934020 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:09.934026 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:09.935951 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
W20260808 04:59:09.936096 20034 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:09.931158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:36417 (local address 127.3.174.190:35877)
0808 04:59:09.931259 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:09.931261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:09.931286 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:59:09.931349 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:09.931350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:09.931361 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:09.931411 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:09.931731 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.932251 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.933231 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:09.933494 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:09.933544 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:09.936042 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:36417: BlockingRecv error: recv got EOF from 127.3.174.131:36417 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 04:59:10 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:10 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:10.703644 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:10.699311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:42163)
0808 04:59:10.699451 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:10.699462 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:10.699516 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:10.699656 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:10.699657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:10.699842 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:10.699893 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:10.699894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.700526 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.700526 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:10.701043 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:10.701044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.701405 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.701405 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:10.701447 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:10.701857 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:10.701861 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:10.703567 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
W20260808 04:59:10.703720 20038 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:10.699392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:42163 (local address 127.3.174.190:35877)
0808 04:59:10.699527 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:10.699529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:10.699551 (+ 22us) server_negotiation.cc:407] Connection header received
0808 04:59:10.699572 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:10.699573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:10.699584 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:10.699630 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:10.699955 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.700467 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.701107 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.701346 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.701404 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:10.703657 (+ 2253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:42163: BlockingRecv error: recv got EOF from 127.3.174.129:42163 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
Aug 08 04:59:10 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:10 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:10.808245 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:10.803735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:55955)
0808 04:59:10.803880 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:10.803892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:10.803953 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:10.804073 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:10.804074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:10.804290 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:10.804339 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:10.804341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.805000 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.805001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:10.805598 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:10.805600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.805966 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.805967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:10.806011 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:10.806432 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:10.806437 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:10.808175 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
W20260808 04:59:10.808346 20038 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:10.803804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:55955 (local address 127.3.174.190:35877)
0808 04:59:10.803867 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:10.803869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:10.803946 (+ 77us) server_negotiation.cc:407] Connection header received
0808 04:59:10.804010 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:10.804011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:10.804022 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:10.804062 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:10.804413 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.804928 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.805687 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.805894 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.805956 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:10.808279 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:55955: BlockingRecv error: recv got EOF from 127.3.174.130:55955 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 04:59:10 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:10 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.131@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:10.941612 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:10.936532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.131:44441)
0808 04:59:10.936689 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:10.936703 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:10.936778 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:10.936933 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:10.936935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:10.937132 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:10.937190 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:10.937193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.937926 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.937928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:10.938465 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:10.938468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.938881 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.938882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:10.938940 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:10.939570 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:10.939575 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:10.941533 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
W20260808 04:59:10.941690 20038 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:10.936644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:44441 (local address 127.3.174.190:35877)
0808 04:59:10.936717 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:10.936719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:10.936756 (+ 37us) server_negotiation.cc:407] Connection header received
0808 04:59:10.936844 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:10.936845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:10.936863 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:10.936919 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:10.937285 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.937836 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.938561 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:10.938794 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:10.938855 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:10.941636 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.131:44441: BlockingRecv error: recv got EOF from 127.3.174.131:44441 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 04:59:11 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:11 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.129@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:11.708772 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:11.704253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.129:34255)
0808 04:59:11.704413 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:11.704424 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:11.704490 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:11.704612 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:11.704613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:11.704799 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:11.704850 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:11.704852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:11.705575 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:11.705577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:11.706110 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:11.706112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:11.706434 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:11.706435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:11.706474 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:11.706890 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:11.706895 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:11.708691 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
W20260808 04:59:11.708914 20042 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:11.704334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:34255 (local address 127.3.174.190:35877)
0808 04:59:11.704438 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:11.704440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:11.704473 (+ 33us) server_negotiation.cc:407] Connection header received
0808 04:59:11.704542 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:11.704543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:11.704554 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:11.704598 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:11.704911 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:11.705502 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:11.706181 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:11.706385 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:11.706430 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:11.708802 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.129:34255: BlockingRecv error: recv got EOF from 127.3.174.129:34255 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":53,"threads_started":1}
Aug 08 04:59:11 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 04:59:11 dist-test-slave-gjw7 krb5kdc[18177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.174.130@KRBTEST.COM for oryx/127.3.174.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 04:59:11.813211 20043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:11.808764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35877 (local address 127.3.174.130:46701)
0808 04:59:11.808916 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:11.808927 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:11.808993 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:11.809119 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:11.809120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:11.809308 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:11.809368 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:11.809370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:11.810010 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:11.810011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:11.810539 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:11.810541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:11.810911 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:11.810912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:11.810963 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:11.811371 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:11.811374 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:11.813150 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.174.190:35877: Server oryx/127.3.174.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
W20260808 04:59:11.813378 20042 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:11.808792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:46701 (local address 127.3.174.190:35877)
0808 04:59:11.808837 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:11.808839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:11.808987 (+ 148us) server_negotiation.cc:407] Connection header received
0808 04:59:11.809054 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:11.809055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:11.809067 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:11.809108 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:11.809428 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:11.809926 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:11.810616 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:11.810819 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:11.810879 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:11.813316 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.174.130:46701: BlockingRecv error: recv got EOF from 127.3.174.130:46701 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 04:59:11.861744 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 19209
I20260808 04:59:11.880076 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 19482
I20260808 04:59:11.899103 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 19712
I20260808 04:59:11.919286 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 19091
2026-08-08T04:59:11Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17663 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T04:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:59:11Z Disabled control of system clock
I20260808 04:59:11.952553 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33165
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:35851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:33165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 04:59:12.035068 20050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:12.035272 20050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:12.035293 20050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:12.036461 20050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:59:12.036499 20050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:12.036510 20050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:12.036521 20050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:59:12.036535 20050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:59:12.037878 20050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:33165
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:12.038105 20050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:12.038378 20050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:12.043100 20063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.043147 20064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.043339 20068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:12.043219 20050 server_base.cc:1034] running on GCE node
I20260808 04:59:12.043606 20050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:12.043895 20050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:12.045063 20050 hybrid_clock.cc:648] HybridClock initialized: now 1786165152045049 us; error 25 us; skew 500 ppm
I20260808 04:59:12.046269 20050 webserver.cc:460] Webserver started at http://127.3.174.190:42499/ using document root <none> and password file <none>
I20260808 04:59:12.046479 20050 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:12.046523 20050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:12.046615 20050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:12.047622 20050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "0531dcbd44d24c63a2f55e0242994b65"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "267f645d3e87204d483d99ffa5fb3ad3"
server_key_iv: "1dabf7b15ae2a62a4270daa5c6b4d8f2"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.047923 20050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "0531dcbd44d24c63a2f55e0242994b65"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "267f645d3e87204d483d99ffa5fb3ad3"
server_key_iv: "1dabf7b15ae2a62a4270daa5c6b4d8f2"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.049175 20050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:59:12.049944 20076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.050100 20050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:59:12.050212 20050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "0531dcbd44d24c63a2f55e0242994b65"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "267f645d3e87204d483d99ffa5fb3ad3"
server_key_iv: "1dabf7b15ae2a62a4270daa5c6b4d8f2"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.050277 20050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:12.065685 20050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:12.066012 20050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:12.066151 20050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:12.070098 20050 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:33165
I20260808 04:59:12.070127 20172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:33165 every 8 connection(s)
I20260808 04:59:12.070483 20050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:59:12.071208 20173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:59:12.073926 20173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65: Bootstrap starting.
I20260808 04:59:12.074541 20173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65: Neither blocks nor log segments found. Creating new log.
I20260808 04:59:12.074816 20173 log.cc:824] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65: Log is configured to *not* fsync() on all Append() calls
I20260808 04:59:12.075487 20173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65: No bootstrap required, opened a new log
I20260808 04:59:12.076805 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 20050
I20260808 04:59:12.076882 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:59:12.077042 3770 external_mini_cluster.cc:1428] Setting key 0c554e7714ad0a676217b3d58fd110f9
I20260808 04:59:12.077102 20173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0531dcbd44d24c63a2f55e0242994b65" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33165 } }
I20260808 04:59:12.077270 20173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:59:12.077332 20173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0531dcbd44d24c63a2f55e0242994b65, State: Initialized, Role: FOLLOWER
I20260808 04:59:12.077500 20173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0531dcbd44d24c63a2f55e0242994b65" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33165 } }
I20260808 04:59:12.077687 20173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:59:12.077769 20173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:59:12.077827 20173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:59:12.078501 20173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0531dcbd44d24c63a2f55e0242994b65" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33165 } }
I20260808 04:59:12.078617 20173 leader_election.cc:302] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0531dcbd44d24c63a2f55e0242994b65; no voters:
I20260808 04:59:12.078778 20173 leader_election.cc:288] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:59:12.078817 20182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:59:12.078981 20182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [term 1 LEADER]: Becoming Leader. State: Replica: 0531dcbd44d24c63a2f55e0242994b65, State: Running, Role: LEADER
I20260808 04:59:12.079030 20173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:59:12.080029 20182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0531dcbd44d24c63a2f55e0242994b65" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33165 } }
I20260808 04:59:12.080670 20183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0531dcbd44d24c63a2f55e0242994b65. Latest consensus state: current_term: 1 leader_uuid: "0531dcbd44d24c63a2f55e0242994b65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0531dcbd44d24c63a2f55e0242994b65" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33165 } } }
I20260808 04:59:12.080681 20186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0531dcbd44d24c63a2f55e0242994b65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0531dcbd44d24c63a2f55e0242994b65" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33165 } } }
I20260808 04:59:12.080761 20183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [sys.catalog]: This master's current role is: LEADER
I20260808 04:59:12.080775 20186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65 [sys.catalog]: This master's current role is: LEADER
I20260808 04:59:12.081002 20205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:59:12.081535 20205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:59:12.082988 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.077571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45982 (local address 127.3.174.190:33165)
0808 04:59:12.077822 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:12.077832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:12.077865 (+ 33us) server_negotiation.cc:407] Connection header received
0808 04:59:12.077999 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:12.078007 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:12.078078 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:12.078232 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:12.078410 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.079966 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.080965 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.081567 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.081680 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:12.082324 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:12.082329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:12.082329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:59:12.082525 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:12.082569 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:59:12.082585 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:12.082644 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:12.082720 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 04:59:12.082799 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":109,"threads_started":1}
I20260808 04:59:12.083016 20205 catalog_manager.cc:1347] Generated new cluster ID: 5ca7759cf1554e0f8244fe424f10a064
I20260808 04:59:12.083065 20205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:59:12.096524 20205 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:59:12.097071 20205 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:59:12.103988 20205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0531dcbd44d24c63a2f55e0242994b65: Generated new TSK 0
I20260808 04:59:12.104161 20205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:59:12.110757 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33165
--builtin_ntp_servers=127.3.174.148:35851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 04:59:12.215754 20211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:12.215955 20211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:12.215975 20211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:12.217115 20211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:12.217154 20211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:12.217177 20211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:59:12.218493 20211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:12.218708 20211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:12.219005 20211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:12.223779 20219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.223789 20225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.223821 20221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:12.224102 20211 server_base.cc:1034] running on GCE node
I20260808 04:59:12.224304 20211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:12.224537 20211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:12.225683 20211 hybrid_clock.cc:648] HybridClock initialized: now 1786165152225670 us; error 27 us; skew 500 ppm
I20260808 04:59:12.226975 20211 webserver.cc:460] Webserver started at http://127.3.174.129:35507/ using document root <none> and password file <none>
I20260808 04:59:12.227259 20211 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:12.227315 20211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:12.227443 20211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:12.228462 20211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "aad0fba69195491eb619d46192327596"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "3a673718fb0622652795eab0c8ae50a5"
server_key_iv: "4ef78cc13f39baffbf37dffbcba64f3e"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.228791 20211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "aad0fba69195491eb619d46192327596"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "3a673718fb0622652795eab0c8ae50a5"
server_key_iv: "4ef78cc13f39baffbf37dffbcba64f3e"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.230105 20211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:59:12.230818 20240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.230973 20211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:59:12.231065 20211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "aad0fba69195491eb619d46192327596"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "3a673718fb0622652795eab0c8ae50a5"
server_key_iv: "4ef78cc13f39baffbf37dffbcba64f3e"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.231129 20211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:12.250114 20211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:12.250394 20211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:12.250512 20211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:12.250715 20211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:59:12.251016 20211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:59:12.251048 20211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.251076 20211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:59:12.251101 20211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.256873 20211 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:34721
I20260808 04:59:12.256923 20443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:34721 every 8 connection(s)
I20260808 04:59:12.257232 20211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:59:12.262392 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.258001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:36207 (local address 127.3.174.190:33165)
0808 04:59:12.258075 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:12.258077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:12.258318 (+ 241us) server_negotiation.cc:407] Connection header received
0808 04:59:12.258827 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:12.258829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:12.258845 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:12.258896 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:12.259582 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.260094 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.261031 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.261235 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.261316 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:12.262072 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:12.262074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:12.262074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:59:12.262079 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:12.262095 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:59:12.262099 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:12.262144 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:12.262275 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 04:59:12.262312 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 04:59:12.262585 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.257974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33165 (local address 127.3.174.129:36207)
0808 04:59:12.258160 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:12.258235 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:12.258734 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:12.258963 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:12.258967 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:12.258976 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:12.259492 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:12.259497 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.260180 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.260187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:12.260945 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:12.260948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.261365 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.261366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:12.261475 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:12.261937 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:59:12.261937 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:12.261966 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:59:12.261989 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:59:12.262150 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:59:12.262156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:59:12.262160 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:59:12.262266 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 04:59:12.262368 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260808 04:59:12.263073 20445 heartbeater.cc:347] Connected to a master server at 127.3.174.190:33165
I20260808 04:59:12.263167 20445 heartbeater.cc:464] Registering TS with master...
I20260808 04:59:12.263365 20445 heartbeater.cc:511] Master 127.3.174.190:33165 requested a full tablet report, sending...
I20260808 04:59:12.263878 20101 ts_manager.cc:194] Registered new tserver with Master: aad0fba69195491eb619d46192327596 (127.3.174.129:34721)
I20260808 04:59:12.264492 20101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.129:36207
I20260808 04:59:12.267269 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 20211
I20260808 04:59:12.267343 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:59:12.267526 3770 external_mini_cluster.cc:1428] Setting key 104d1d32d12c084f0dbfc09ae2847a8f
I20260808 04:59:12.268368 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33165
--builtin_ntp_servers=127.3.174.148:35851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 04:59:12.352044 20448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:12.352232 20448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:12.352258 20448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:12.353443 20448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:12.353482 20448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:12.353504 20448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:59:12.354847 20448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:12.355044 20448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:12.355311 20448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:12.360198 20457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.360165 20455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:12.360249 20448 server_base.cc:1034] running on GCE node
W20260808 04:59:12.360186 20464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:12.360582 20448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:12.360823 20448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:12.361963 20448 hybrid_clock.cc:648] HybridClock initialized: now 1786165152361950 us; error 30 us; skew 500 ppm
I20260808 04:59:12.363229 20448 webserver.cc:460] Webserver started at http://127.3.174.130:35887/ using document root <none> and password file <none>
I20260808 04:59:12.363456 20448 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:12.363505 20448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:12.363633 20448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:12.364665 20448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "33725c1a53774bd981139de525f61a13"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "72eba7ab5206ce7daa378f730daa75e1"
server_key_iv: "7f322db3664fb0ee7dac98c55ccb9eb8"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.365000 20448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "33725c1a53774bd981139de525f61a13"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "72eba7ab5206ce7daa378f730daa75e1"
server_key_iv: "7f322db3664fb0ee7dac98c55ccb9eb8"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.366281 20448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:59:12.366928 20472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.367089 20448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:59:12.367172 20448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "33725c1a53774bd981139de525f61a13"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "72eba7ab5206ce7daa378f730daa75e1"
server_key_iv: "7f322db3664fb0ee7dac98c55ccb9eb8"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.367231 20448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:12.381052 20448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:12.381321 20448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:12.381441 20448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:12.381632 20448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:59:12.381929 20448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:59:12.381964 20448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.381992 20448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:59:12.382016 20448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.387444 20448 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:33173
I20260808 04:59:12.387523 20673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:33173 every 8 connection(s)
I20260808 04:59:12.387786 20448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:59:12.392719 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.388488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:54609 (local address 127.3.174.190:33165)
0808 04:59:12.388569 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:12.388572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:12.388796 (+ 224us) server_negotiation.cc:407] Connection header received
0808 04:59:12.389346 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:12.389347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:12.389361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:12.389422 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:12.390043 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.390605 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.391530 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.391728 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.391791 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:12.392502 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:12.392503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:12.392503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:59:12.392507 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:12.392522 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:59:12.392526 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:12.392568 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:12.392627 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 04:59:12.392651 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 04:59:12.392889 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.388477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33165 (local address 127.3.174.130:54609)
0808 04:59:12.388675 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:12.388713 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:12.389158 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:12.389477 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:12.389481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:12.389490 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:12.389963 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:12.389968 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.390685 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.390686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:12.391437 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:12.391439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.391795 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.391796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:12.391896 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:12.392374 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:59:12.392374 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:12.392402 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:59:12.392423 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:59:12.392573 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:59:12.392576 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:59:12.392578 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:59:12.392622 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 04:59:12.392699 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260808 04:59:12.393402 20674 heartbeater.cc:347] Connected to a master server at 127.3.174.190:33165
I20260808 04:59:12.393481 20674 heartbeater.cc:464] Registering TS with master...
I20260808 04:59:12.393669 20674 heartbeater.cc:511] Master 127.3.174.190:33165 requested a full tablet report, sending...
I20260808 04:59:12.393718 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 20448
I20260808 04:59:12.393805 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:59:12.393980 3770 external_mini_cluster.cc:1428] Setting key 58c18d81782ce457801da55927805fcb
I20260808 04:59:12.394073 20101 ts_manager.cc:194] Registered new tserver with Master: 33725c1a53774bd981139de525f61a13 (127.3.174.130:33173)
I20260808 04:59:12.394665 20101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.130:54609
I20260808 04:59:12.394964 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33165
--builtin_ntp_servers=127.3.174.148:35851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 04:59:12.476815 20678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:12.477008 20678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:12.477027 20678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:12.478124 20678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:12.478156 20678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:12.478179 20678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:59:12.479506 20678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:12.479698 20678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:12.480021 20678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:12.484797 20690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.484812 20693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:12.484869 20678 server_base.cc:1034] running on GCE node
W20260808 04:59:12.484799 20696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:12.485244 20678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:12.485543 20678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:12.486683 20678 hybrid_clock.cc:648] HybridClock initialized: now 1786165152486670 us; error 28 us; skew 500 ppm
I20260808 04:59:12.487854 20678 webserver.cc:460] Webserver started at http://127.3.174.131:33709/ using document root <none> and password file <none>
I20260808 04:59:12.488073 20678 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:12.488123 20678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:12.488255 20678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:12.489344 20678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "434e543f567349d3828af218255cb99d"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "0b9cd125554eec4da04da1dcd5a8f321"
server_key_iv: "8a3578239a0fc5064e1dedfac1822098"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.489693 20678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "434e543f567349d3828af218255cb99d"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "0b9cd125554eec4da04da1dcd5a8f321"
server_key_iv: "8a3578239a0fc5064e1dedfac1822098"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.490952 20678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:59:12.491606 20709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.491770 20678 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:59:12.491853 20678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "434e543f567349d3828af218255cb99d"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "0b9cd125554eec4da04da1dcd5a8f321"
server_key_iv: "8a3578239a0fc5064e1dedfac1822098"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.491920 20678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:12.507397 20678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:12.507700 20678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:12.507831 20678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:12.508014 20678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:59:12.508311 20678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:59:12.508345 20678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.508366 20678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:59:12.508387 20678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.513847 20678 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:35255
I20260808 04:59:12.513891 20894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:35255 every 8 connection(s)
I20260808 04:59:12.514189 20678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 04:59:12.519312 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.514954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:46781 (local address 127.3.174.190:33165)
0808 04:59:12.515009 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:12.515012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:12.515255 (+ 243us) server_negotiation.cc:407] Connection header received
0808 04:59:12.515713 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:12.515715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:12.515725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:12.515777 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:12.516416 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.516917 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.517843 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.518037 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.518088 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:12.518928 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:12.518929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:12.518929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:59:12.518933 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:12.518950 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:59:12.518954 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:12.518999 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:12.519150 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 04:59:12.519247 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 04:59:12.519361 20897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.514938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33165 (local address 127.3.174.131:46781)
0808 04:59:12.515119 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:12.515153 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:12.515627 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:12.515834 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:12.515837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:12.515846 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:12.516335 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:12.516340 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.516980 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.516981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:12.517772 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:12.517774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:12.518132 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:12.518134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:12.518276 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:12.518791 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:59:12.518792 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:12.518821 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:59:12.518844 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:59:12.519028 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:59:12.519034 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:59:12.519038 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:59:12.519107 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 04:59:12.519200 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260808 04:59:12.519906 20895 heartbeater.cc:347] Connected to a master server at 127.3.174.190:33165
I20260808 04:59:12.520004 20895 heartbeater.cc:464] Registering TS with master...
I20260808 04:59:12.520243 20895 heartbeater.cc:511] Master 127.3.174.190:33165 requested a full tablet report, sending...
I20260808 04:59:12.520403 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 20678
I20260808 04:59:12.520495 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:59:12.520689 20101 ts_manager.cc:194] Registered new tserver with Master: 434e543f567349d3828af218255cb99d (127.3.174.131:35255)
I20260808 04:59:12.520720 3770 external_mini_cluster.cc:1428] Setting key 21b6fb0f7f64c6678a678bf6ff82d90b
I20260808 04:59:12.521057 20101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.131:46781
I20260808 04:59:12.521696 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 04:59:12.523546 20179 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:12.523054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.3.174.190:33165)
0808 04:59:12.523112 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:12.523114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:12.523216 (+ 102us) server_negotiation.cc:407] Connection header received
0808 04:59:12.523246 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:12.523246 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:12.523260 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:12.523318 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:12.523483 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45996: BlockingRecv error: recv got EOF from 127.0.0.1:45996 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:59:12.523572 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 20211
I20260808 04:59:12.542207 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 20448
I20260808 04:59:12.553649 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 20678
I20260808 04:59:12.569437 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 20050
2026-08-08T04:59:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (646 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T04:59:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:59:12Z Disabled control of system clock
I20260808 04:59:12.599267 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:44893
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:44893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 04:59:12.680332 20909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:12.680524 20909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:12.680543 20909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:12.681663 20909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:59:12.681701 20909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:12.681712 20909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:12.681723 20909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:59:12.681733 20909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:59:12.683028 20909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:44893
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:44893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:12.683233 20909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:12.683466 20909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:12.688086 20919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.688125 20918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.688083 20921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:12.688445 20909 server_base.cc:1034] running on GCE node
I20260808 04:59:12.688690 20909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:12.688954 20909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:12.690089 20909 hybrid_clock.cc:648] HybridClock initialized: now 1786165152690060 us; error 49 us; skew 500 ppm
I20260808 04:59:12.691325 20909 webserver.cc:460] Webserver started at http://127.3.174.190:39297/ using document root <none> and password file <none>
I20260808 04:59:12.691566 20909 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:12.691613 20909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:12.691741 20909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:12.692778 20909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "1da9f6b75663476587aacecc1457c59c"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "607cc0ed3b789c2fd6703b2293572b1f"
server_key_iv: "9c30430a76a3ec708b044d18cebd0aef"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.693172 20909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "1da9f6b75663476587aacecc1457c59c"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "607cc0ed3b789c2fd6703b2293572b1f"
server_key_iv: "9c30430a76a3ec708b044d18cebd0aef"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.694458 20909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 04:59:12.695101 20932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.695300 20909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:59:12.695382 20909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "1da9f6b75663476587aacecc1457c59c"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "607cc0ed3b789c2fd6703b2293572b1f"
server_key_iv: "9c30430a76a3ec708b044d18cebd0aef"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.695448 20909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:12.699002 20909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:12.699246 20909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:12.699370 20909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:12.703107 20909 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:44893
I20260808 04:59:12.703156 21022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:44893 every 8 connection(s)
I20260808 04:59:12.703483 20909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:59:12.704188 21023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:59:12.707082 21023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c: Bootstrap starting.
I20260808 04:59:12.707609 21023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c: Neither blocks nor log segments found. Creating new log.
I20260808 04:59:12.707840 21023 log.cc:824] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c: Log is configured to *not* fsync() on all Append() calls
I20260808 04:59:12.708520 21023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c: No bootstrap required, opened a new log
I20260808 04:59:12.710155 21023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9f6b75663476587aacecc1457c59c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 44893 } }
I20260808 04:59:12.710299 21023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:59:12.710320 21023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da9f6b75663476587aacecc1457c59c, State: Initialized, Role: FOLLOWER
I20260808 04:59:12.710436 21023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9f6b75663476587aacecc1457c59c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 44893 } }
I20260808 04:59:12.710515 21023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:59:12.710543 21023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:59:12.710575 21023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:59:12.711052 21023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9f6b75663476587aacecc1457c59c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 44893 } }
I20260808 04:59:12.711143 21023 leader_election.cc:302] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1da9f6b75663476587aacecc1457c59c; no voters:
I20260808 04:59:12.711277 21023 leader_election.cc:288] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:59:12.711335 21028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:59:12.711505 21028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [term 1 LEADER]: Becoming Leader. State: Replica: 1da9f6b75663476587aacecc1457c59c, State: Running, Role: LEADER
I20260808 04:59:12.711503 21023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:59:12.711617 21028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9f6b75663476587aacecc1457c59c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 44893 } }
I20260808 04:59:12.712244 21031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1da9f6b75663476587aacecc1457c59c. Latest consensus state: current_term: 1 leader_uuid: "1da9f6b75663476587aacecc1457c59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9f6b75663476587aacecc1457c59c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 44893 } } }
I20260808 04:59:12.712343 21031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [sys.catalog]: This master's current role is: LEADER
I20260808 04:59:12.712617 21041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:59:12.712716 21029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1da9f6b75663476587aacecc1457c59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9f6b75663476587aacecc1457c59c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 44893 } } }
I20260808 04:59:12.712785 21029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c [sys.catalog]: This master's current role is: LEADER
I20260808 04:59:12.713156 21041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:59:12.713425 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 20909
I20260808 04:59:12.713518 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:59:12.713698 3770 external_mini_cluster.cc:1428] Setting key 4a56eac71152b605fc5a1108b97d0135
I20260808 04:59:12.715770 21041 catalog_manager.cc:1347] Generated new cluster ID: 3a9eafea96e94f70ae9758f23ebab1d3
I20260808 04:59:12.715827 21041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:59:12.716015 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.714141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41552 (local address 127.3.174.190:44893)
0808 04:59:12.714351 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:12.714360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:12.714396 (+ 36us) server_negotiation.cc:407] Connection header received
0808 04:59:12.714511 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:12.714519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:12.714577 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:12.714706 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:12.715393 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:12.715396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:12.715397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:59:12.715401 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:12.715451 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:59:12.715467 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:12.715528 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:12.715794 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 04:59:12.715853 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260808 04:59:12.745491 21041 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:59:12.745678 21041 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:59:12.753409 21041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1da9f6b75663476587aacecc1457c59c: Generated new TSK 0
I20260808 04:59:12.753592 21041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 04:59:12.755457 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:44893
--builtin_ntp_servers=127.3.174.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 04:59:12.839744 21059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:12.839946 21059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:12.839975 21059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:12.841142 21059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:12.841180 21059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:12.841214 21059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:59:12.842571 21059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:44893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:12.842821 21059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:12.843101 21059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:12.848004 21070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.848032 21072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.848250 21080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:12.848248 21059 server_base.cc:1034] running on GCE node
I20260808 04:59:12.848455 21059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:12.848694 21059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:12.849839 21059 hybrid_clock.cc:648] HybridClock initialized: now 1786165152849819 us; error 35 us; skew 500 ppm
I20260808 04:59:12.850983 21059 webserver.cc:460] Webserver started at http://127.3.174.129:42763/ using document root <none> and password file <none>
I20260808 04:59:12.851245 21059 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:12.851297 21059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:12.851406 21059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:12.852542 21059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "5fa2528d93cb4ac78d03b46ebb2c0bfd"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "144326e923f693f2a80c3a54da2df35b"
server_key_iv: "0e53d028f563db6557b67ae527224935"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.852851 21059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "5fa2528d93cb4ac78d03b46ebb2c0bfd"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "144326e923f693f2a80c3a54da2df35b"
server_key_iv: "0e53d028f563db6557b67ae527224935"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.854231 21059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:59:12.854910 21094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.855062 21059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:59:12.855145 21059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "5fa2528d93cb4ac78d03b46ebb2c0bfd"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "144326e923f693f2a80c3a54da2df35b"
server_key_iv: "0e53d028f563db6557b67ae527224935"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.855207 21059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:12.858455 21059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:12.858682 21059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:12.858808 21059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:12.858983 21059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:59:12.859254 21059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:59:12.859285 21059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.859310 21059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:59:12.859335 21059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.864647 21059 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:42207
I20260808 04:59:12.864705 21275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:42207 every 8 connection(s)
I20260808 04:59:12.864969 21059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 04:59:12.867535 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.865631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:59561 (local address 127.3.174.190:44893)
0808 04:59:12.865720 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:12.865722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:12.865898 (+ 176us) server_negotiation.cc:407] Connection header received
0808 04:59:12.866300 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:12.866301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:12.866309 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:12.866359 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:12.867356 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:12.867357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:12.867357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:59:12.867361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:12.867376 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:59:12.867380 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:12.867427 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:12.867471 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 04:59:12.867484 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 04:59:12.867695 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.865591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:44893 (local address 127.3.174.129:59561)
0808 04:59:12.865774 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:12.865811 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:12.866225 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:12.866420 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:12.866424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:12.866432 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:12.867221 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:59:12.867221 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:12.867247 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:59:12.867274 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:59:12.867425 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:59:12.867427 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:59:12.867429 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:59:12.867472 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 04:59:12.867518 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260808 04:59:12.868115 21276 heartbeater.cc:347] Connected to a master server at 127.3.174.190:44893
I20260808 04:59:12.868199 21276 heartbeater.cc:464] Registering TS with master...
I20260808 04:59:12.868305 21276 heartbeater.cc:511] Master 127.3.174.190:44893 requested a full tablet report, sending...
I20260808 04:59:12.868768 20962 ts_manager.cc:194] Registered new tserver with Master: 5fa2528d93cb4ac78d03b46ebb2c0bfd (127.3.174.129:42207)
I20260808 04:59:12.871105 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 21059
I20260808 04:59:12.871233 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:59:12.871431 3770 external_mini_cluster.cc:1428] Setting key 3e690cc309dcb9d88226107ef007d971
I20260808 04:59:12.872305 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:44893
--builtin_ntp_servers=127.3.174.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 04:59:12.953418 21279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:12.953619 21279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:12.953647 21279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:12.954869 21279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:12.954909 21279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:12.954943 21279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:59:12.956538 21279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:44893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:12.956784 21279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:12.957054 21279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 04:59:12.961900 21279 server_base.cc:1034] running on GCE node
W20260808 04:59:12.961943 21291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.961954 21288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:12.962010 21289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:12.962407 21279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:12.962656 21279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:12.963816 21279 hybrid_clock.cc:648] HybridClock initialized: now 1786165152963801 us; error 30 us; skew 500 ppm
I20260808 04:59:12.965054 21279 webserver.cc:460] Webserver started at http://127.3.174.130:34299/ using document root <none> and password file <none>
I20260808 04:59:12.965341 21279 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:12.965394 21279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:12.965519 21279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:12.966609 21279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "c4c945e57c844cd5b6ee2f0c5fba6053"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "dbd5ffe5d0c271373ca5e68176b82321"
server_key_iv: "890f79ae87ba2882e19f02939556b2fc"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.966919 21279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "c4c945e57c844cd5b6ee2f0c5fba6053"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "dbd5ffe5d0c271373ca5e68176b82321"
server_key_iv: "890f79ae87ba2882e19f02939556b2fc"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.968365 21279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 04:59:12.969094 21302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.969282 21279 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:59:12.969400 21279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "c4c945e57c844cd5b6ee2f0c5fba6053"
format_stamp: "Formatted at 2026-08-08 04:59:12 on dist-test-slave-gjw7"
server_key: "dbd5ffe5d0c271373ca5e68176b82321"
server_key_iv: "890f79ae87ba2882e19f02939556b2fc"
server_key_version: "encryptionkey@0"
I20260808 04:59:12.969470 21279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:12.972635 21279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:12.972865 21279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:12.972981 21279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:12.973155 21279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:59:12.973490 21279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:59:12.973526 21279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.973551 21279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:59:12.973575 21279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:12.979506 21279 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:35085
I20260808 04:59:12.979586 21498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:35085 every 8 connection(s)
I20260808 04:59:12.979854 21279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 04:59:12.982564 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.980561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:55725 (local address 127.3.174.190:44893)
0808 04:59:12.980638 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:12.980641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:12.980817 (+ 176us) server_negotiation.cc:407] Connection header received
0808 04:59:12.981335 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:12.981337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:12.981344 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:12.981408 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:12.982390 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:12.982392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:12.982392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:59:12.982396 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:12.982411 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:59:12.982415 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:12.982457 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:12.982501 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 04:59:12.982513 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 04:59:12.982734 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:12.980535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:44893 (local address 127.3.174.130:55725)
0808 04:59:12.980711 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:12.980744 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:12.981151 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:12.981463 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:12.981466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:12.981475 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:12.982266 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:59:12.982266 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:12.982292 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:59:12.982318 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:59:12.982456 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:59:12.982458 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:59:12.982461 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:59:12.982506 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 04:59:12.982551 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260808 04:59:12.983203 21499 heartbeater.cc:347] Connected to a master server at 127.3.174.190:44893
I20260808 04:59:12.983289 21499 heartbeater.cc:464] Registering TS with master...
I20260808 04:59:12.983430 21499 heartbeater.cc:511] Master 127.3.174.190:44893 requested a full tablet report, sending...
I20260808 04:59:12.983783 20962 ts_manager.cc:194] Registered new tserver with Master: c4c945e57c844cd5b6ee2f0c5fba6053 (127.3.174.130:35085)
I20260808 04:59:12.987367 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 21279
I20260808 04:59:12.987445 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:59:12.987601 3770 external_mini_cluster.cc:1428] Setting key f1ffd5cffae85b1d168fccab5c92090b
I20260808 04:59:12.988663 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:44893
--builtin_ntp_servers=127.3.174.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 04:59:13.071997 21503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:13.072228 21503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:13.072252 21503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:13.073438 21503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:13.073480 21503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:13.073508 21503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:59:13.074869 21503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:44893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:13.075104 21503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:13.075402 21503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:13.080156 21512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:13.080163 21514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:13.080384 21503 server_base.cc:1034] running on GCE node
W20260808 04:59:13.080170 21516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:13.080646 21503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:13.080842 21503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:13.081979 21503 hybrid_clock.cc:648] HybridClock initialized: now 1786165153081962 us; error 33 us; skew 500 ppm
I20260808 04:59:13.083233 21503 webserver.cc:460] Webserver started at http://127.3.174.131:44029/ using document root <none> and password file <none>
I20260808 04:59:13.083429 21503 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:13.083467 21503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:13.083556 21503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:13.084569 21503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "a846064a80704f5f8f452c686587e2fb"
format_stamp: "Formatted at 2026-08-08 04:59:13 on dist-test-slave-gjw7"
server_key: "94229fa9e70184f192b88bb7f3f1b803"
server_key_iv: "0b4ff152b03d773ae2bc2df3f5c69c18"
server_key_version: "encryptionkey@0"
I20260808 04:59:13.084894 21503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "a846064a80704f5f8f452c686587e2fb"
format_stamp: "Formatted at 2026-08-08 04:59:13 on dist-test-slave-gjw7"
server_key: "94229fa9e70184f192b88bb7f3f1b803"
server_key_iv: "0b4ff152b03d773ae2bc2df3f5c69c18"
server_key_version: "encryptionkey@0"
I20260808 04:59:13.086231 21503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 04:59:13.086902 21528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:13.087062 21503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:59:13.087142 21503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "a846064a80704f5f8f452c686587e2fb"
format_stamp: "Formatted at 2026-08-08 04:59:13 on dist-test-slave-gjw7"
server_key: "94229fa9e70184f192b88bb7f3f1b803"
server_key_iv: "0b4ff152b03d773ae2bc2df3f5c69c18"
server_key_version: "encryptionkey@0"
I20260808 04:59:13.087234 21503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:13.090425 21503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:13.090622 21503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:13.090731 21503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:13.090888 21503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:59:13.091120 21503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:59:13.091148 21503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:13.091166 21503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:59:13.091190 21503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:13.096421 21503 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:41077
I20260808 04:59:13.096481 21708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:41077 every 8 connection(s)
I20260808 04:59:13.096732 21503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 04:59:13.099399 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:13.097373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:47863 (local address 127.3.174.190:44893)
0808 04:59:13.097440 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:13.097442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:13.097627 (+ 185us) server_negotiation.cc:407] Connection header received
0808 04:59:13.098076 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:13.098077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:13.098084 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:13.098132 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:13.099147 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:13.099149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:13.099149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 04:59:13.099153 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:13.099168 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 04:59:13.099172 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:13.099253 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:13.099305 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 04:59:13.099317 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 04:59:13.099545 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:13.097365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:44893 (local address 127.3.174.131:47863)
0808 04:59:13.097526 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:13.097557 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:13.097980 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:13.098184 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:13.098188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:13.098197 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:13.099014 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 04:59:13.099014 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:13.099041 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 04:59:13.099069 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:59:13.099257 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:59:13.099260 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:59:13.099262 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 04:59:13.099310 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 04:59:13.099362 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260808 04:59:13.099985 21709 heartbeater.cc:347] Connected to a master server at 127.3.174.190:44893
I20260808 04:59:13.100070 21709 heartbeater.cc:464] Registering TS with master...
I20260808 04:59:13.100214 21709 heartbeater.cc:511] Master 127.3.174.190:44893 requested a full tablet report, sending...
I20260808 04:59:13.100553 20962 ts_manager.cc:194] Registered new tserver with Master: a846064a80704f5f8f452c686587e2fb (127.3.174.131:41077)
I20260808 04:59:13.103945 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 21503
I20260808 04:59:13.104022 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:59:13.104187 3770 external_mini_cluster.cc:1428] Setting key be08b583cd2baedbb892a19dd9db9229
I20260808 04:59:13.104945 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 04:59:13.106593 21057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 04:59:13.106228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.3.174.190:44893)
0808 04:59:13.106277 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:13.106278 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:13.106356 (+ 78us) server_negotiation.cc:407] Connection header received
0808 04:59:13.106410 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:13.106411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:13.106415 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:13.106463 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:13.106547 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41568: BlockingRecv error: recv got EOF from 127.0.0.1:41568 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 04:59:13.106683 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 21059
I20260808 04:59:13.126026 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 21279
I20260808 04:59:13.131496 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 21503
I20260808 04:59:13.151687 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 20909
2026-08-08T04:59:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (592 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:59:13 dist-test-slave-gjw7 krb5kdc[21722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:59:13 dist-test-slave-gjw7 krb5kdc[21722](info): set up 2 sockets
Aug 08 04:59:13 dist-test-slave-gjw7 krb5kdc[21722](info): commencing operation
krb5kdc: starting...
W20260808 04:59:14.341540 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:59:14 dist-test-slave-gjw7 krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:59:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:59:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 04:59:14.458253 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46659
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:46659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:59:14.540843 21743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:14.541059 21743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:14.541090 21743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:14.542271 21743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 04:59:14.542313 21743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:14.542331 21743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:14.542348 21743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 04:59:14.542363 21743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 04:59:14.543835 21743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:46659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:14.544104 21743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:14.544307 21743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:14.549017 21756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:14.548983 21752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:14.548992 21754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:14.549204 21743 server_base.cc:1034] running on GCE node
I20260808 04:59:14.549510 21743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:14.549779 21743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:14.550935 21743 hybrid_clock.cc:648] HybridClock initialized: now 1786165154550915 us; error 33 us; skew 500 ppm
Aug 08 04:59:14 dist-test-slave-gjw7 krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165154, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:59:14.556185 21743 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 04:59:14.556624 21743 webserver.cc:460] Webserver started at http://127.3.174.190:33789/ using document root <none> and password file <none>
I20260808 04:59:14.556846 21743 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:14.556898 21743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:14.557060 21743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:14.558185 21743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "2ab71fe0cf6f4de892f6aae0172ead62"
format_stamp: "Formatted at 2026-08-08 04:59:14 on dist-test-slave-gjw7"
server_key: "2caabca7623b83b13941c903bce13ee0"
server_key_iv: "f13e20016a3e89e2532ae8d9ad2375ab"
server_key_version: "encryptionkey@0"
I20260808 04:59:14.558527 21743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "2ab71fe0cf6f4de892f6aae0172ead62"
format_stamp: "Formatted at 2026-08-08 04:59:14 on dist-test-slave-gjw7"
server_key: "2caabca7623b83b13941c903bce13ee0"
server_key_iv: "f13e20016a3e89e2532ae8d9ad2375ab"
server_key_version: "encryptionkey@0"
I20260808 04:59:14.559844 21743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 04:59:14.560537 21777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:14.560674 21743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:14.560761 21743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "2ab71fe0cf6f4de892f6aae0172ead62"
format_stamp: "Formatted at 2026-08-08 04:59:14 on dist-test-slave-gjw7"
server_key: "2caabca7623b83b13941c903bce13ee0"
server_key_iv: "f13e20016a3e89e2532ae8d9ad2375ab"
server_key_version: "encryptionkey@0"
I20260808 04:59:14.560829 21743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:14.573251 21743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:14.575058 21743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:14.575188 21743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:14.579047 21743 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:46659
I20260808 04:59:14.579082 21864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:46659 every 8 connection(s)
I20260808 04:59:14.579417 21743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 04:59:14.580410 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:59:14.582666 21865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62: Bootstrap starting.
I20260808 04:59:14.583108 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 21743
I20260808 04:59:14.583209 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 04:59:14.583348 21865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62: Neither blocks nor log segments found. Creating new log.
I20260808 04:59:14.583411 3770 external_mini_cluster.cc:1428] Setting key 0680968d4811a99b136be32996cb14ca
I20260808 04:59:14.583662 21865 log.cc:824] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62: Log is configured to *not* fsync() on all Append() calls
I20260808 04:59:14.584362 21865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62: No bootstrap required, opened a new log
I20260808 04:59:14.586241 21865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71fe0cf6f4de892f6aae0172ead62" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46659 } }
I20260808 04:59:14.586390 21865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:59:14.586426 21865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab71fe0cf6f4de892f6aae0172ead62, State: Initialized, Role: FOLLOWER
I20260808 04:59:14.586556 21865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71fe0cf6f4de892f6aae0172ead62" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46659 } }
I20260808 04:59:14.586637 21865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 04:59:14.586658 21865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 04:59:14.586704 21865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:59:14.587198 21865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71fe0cf6f4de892f6aae0172ead62" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46659 } }
I20260808 04:59:14.587301 21865 leader_election.cc:302] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ab71fe0cf6f4de892f6aae0172ead62; no voters:
I20260808 04:59:14.587464 21865 leader_election.cc:288] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 04:59:14.587493 21878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:59:14.587598 21878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [term 1 LEADER]: Becoming Leader. State: Replica: 2ab71fe0cf6f4de892f6aae0172ead62, State: Running, Role: LEADER
I20260808 04:59:14.587682 21878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71fe0cf6f4de892f6aae0172ead62" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46659 } }
I20260808 04:59:14.587723 21865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 04:59:14.588027 21878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ab71fe0cf6f4de892f6aae0172ead62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71fe0cf6f4de892f6aae0172ead62" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46659 } } }
I20260808 04:59:14.588104 21878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [sys.catalog]: This master's current role is: LEADER
I20260808 04:59:14.588233 21878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ab71fe0cf6f4de892f6aae0172ead62. Latest consensus state: current_term: 1 leader_uuid: "2ab71fe0cf6f4de892f6aae0172ead62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71fe0cf6f4de892f6aae0172ead62" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46659 } } }
I20260808 04:59:14.588291 21878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62 [sys.catalog]: This master's current role is: LEADER
Aug 08 04:59:14 dist-test-slave-gjw7 krb5kdc[21722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
W20260808 04:59:14.590031 21904 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 04:59:14.590077 21904 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 04:59:14.590128 21886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 04:59:14.590274 21886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 04:59:14.591753 21886 catalog_manager.cc:1347] Generated new cluster ID: b9a336c29fe44c94979ffc273ec03ba4
I20260808 04:59:14.591802 21886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 04:59:14.594151 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:14.583946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.3.174.190:46659)
0808 04:59:14.584117 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:14.584127 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:14.584153 (+ 26us) server_negotiation.cc:407] Connection header received
0808 04:59:14.584627 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:14.584635 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:14.584687 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:14.584811 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:14.585493 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.586318 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.587143 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.587634 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.587734 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:14.590549 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:14.590553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:14.590553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:59:14.590565 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:14.592462 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:14.592887 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:14.592890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:14.592891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:14.592990 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:14.593187 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:14.593187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:14.593187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:14.593491 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:14.593650 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:14.593806 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 04:59:14.593944 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260808 04:59:14.599521 21886 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 04:59:14.599999 21886 catalog_manager.cc:1504] Loading token signing keys...
I20260808 04:59:14.605152 21886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62: Generated new TSK 0
I20260808 04:59:14.605345 21886 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 04:59:14.633495 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46659
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:59:14.716188 21910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:14.716399 21910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:14.716431 21910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:14.717588 21910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:14.717629 21910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:14.717664 21910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 04:59:14.719147 21910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:14.719388 21910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:14.719592 21910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:14.724371 21920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:14.724498 21910 server_base.cc:1034] running on GCE node
W20260808 04:59:14.724380 21922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:14.724382 21917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:14.724773 21910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:14.725001 21910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:14.726135 21910 hybrid_clock.cc:648] HybridClock initialized: now 1786165154726123 us; error 29 us; skew 500 ppm
Aug 08 04:59:14 dist-test-slave-gjw7 krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165154, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:59:14.728631 21910 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 04:59:14.729040 21910 webserver.cc:460] Webserver started at http://127.3.174.129:45487/ using document root <none> and password file <none>
I20260808 04:59:14.729257 21910 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:14.729328 21910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:14.729465 21910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:14.730451 21910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "93ba0efaaf024444941486579832bc21"
format_stamp: "Formatted at 2026-08-08 04:59:14 on dist-test-slave-gjw7"
server_key: "bd4fd02998540bbd58dd62694ad0e31a"
server_key_iv: "45730bcffc9ddf3b2af812169955b43c"
server_key_version: "encryptionkey@0"
I20260808 04:59:14.730777 21910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "93ba0efaaf024444941486579832bc21"
format_stamp: "Formatted at 2026-08-08 04:59:14 on dist-test-slave-gjw7"
server_key: "bd4fd02998540bbd58dd62694ad0e31a"
server_key_iv: "45730bcffc9ddf3b2af812169955b43c"
server_key_version: "encryptionkey@0"
I20260808 04:59:14.732010 21910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 04:59:14.732643 21931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:14.732771 21910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:59:14.732841 21910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "93ba0efaaf024444941486579832bc21"
format_stamp: "Formatted at 2026-08-08 04:59:14 on dist-test-slave-gjw7"
server_key: "bd4fd02998540bbd58dd62694ad0e31a"
server_key_iv: "45730bcffc9ddf3b2af812169955b43c"
server_key_version: "encryptionkey@0"
I20260808 04:59:14.732918 21910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:14.741379 21910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:14.743348 21910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:14.743474 21910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:14.743670 21910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:59:14.743959 21910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:59:14.743992 21910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:14.744020 21910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:59:14.744043 21910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:14.749903 21910 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:46373
I20260808 04:59:14.749939 22111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:46373 every 8 connection(s)
I20260808 04:59:14.750231 21910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 04:59:14 dist-test-slave-gjw7 krb5kdc[21722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165154, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:59:14.758605 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 21910
I20260808 04:59:14.758714 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 04:59:14.758908 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:14.750898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:34315 (local address 127.3.174.190:46659)
0808 04:59:14.750942 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:14.750944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:14.751138 (+ 194us) server_negotiation.cc:407] Connection header received
0808 04:59:14.751654 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:14.751655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:14.751668 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:14.751722 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:14.752651 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.753152 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.754169 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.754375 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.754431 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:14.756423 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:14.756425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:14.756425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:59:14.756432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:14.757588 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:14.757931 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:14.757931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:14.757932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:14.757977 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:14.758226 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:14.758228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:14.758228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:14.758352 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:14.758429 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:14.758752 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 04:59:14.758837 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 04:59:14.758934 3770 external_mini_cluster.cc:1428] Setting key 9765fa03b27e219772f7484360fac930
I20260808 04:59:14.759099 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:14.750856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46659 (local address 127.3.174.129:34315)
0808 04:59:14.751028 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:14.751063 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:14.751554 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:14.751814 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:14.751819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:14.752355 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:14.752558 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:14.752562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.753239 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.753240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:14.754084 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:14.754087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.754445 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.754446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:14.754573 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:14.754969 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:14.754977 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:14.756284 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:59:14.757652 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:59:14.757658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:59:14.757660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:59:14.757864 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:59:14.758021 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:59:14.758021 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:59:14.758021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:59:14.758110 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:59:14.758425 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:59:14.758430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:59:14.758587 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 04:59:14.758705 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 04:59:14.758864 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260808 04:59:14.759562 22113 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46659
I20260808 04:59:14.759662 22113 heartbeater.cc:464] Registering TS with master...
I20260808 04:59:14.759832 22113 heartbeater.cc:511] Master 127.3.174.190:46659 requested a full tablet report, sending...
I20260808 04:59:14.760392 21810 ts_manager.cc:194] Registered new tserver with Master: 93ba0efaaf024444941486579832bc21 (127.3.174.129:46373)
I20260808 04:59:14.761010 21810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:34315
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 04:59:14.782228 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46659
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:59:14.864355 22118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:14.864548 22118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:14.864569 22118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:14.865689 22118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:14.865727 22118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:14.865751 22118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 04:59:14.867159 22118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:14.867358 22118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:14.867528 22118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:14.872287 22128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:14.872311 22130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:14.872300 22127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:14.872622 22118 server_base.cc:1034] running on GCE node
I20260808 04:59:14.872812 22118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:14.873027 22118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:14.874161 22118 hybrid_clock.cc:648] HybridClock initialized: now 1786165154874146 us; error 34 us; skew 500 ppm
Aug 08 04:59:14 dist-test-slave-gjw7 krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165154, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:59:14.876529 22118 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 04:59:14.876946 22118 webserver.cc:460] Webserver started at http://127.3.174.130:35183/ using document root <none> and password file <none>
I20260808 04:59:14.877138 22118 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:14.877187 22118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:14.877336 22118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:14.878333 22118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "a2c312b904a64137a4af23fbfc65e015"
format_stamp: "Formatted at 2026-08-08 04:59:14 on dist-test-slave-gjw7"
server_key: "2cbf8a2552c3ddac233ebdc3b0944ee3"
server_key_iv: "9037202a03feecc2eec856320f78ff1d"
server_key_version: "encryptionkey@0"
I20260808 04:59:14.878707 22118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "a2c312b904a64137a4af23fbfc65e015"
format_stamp: "Formatted at 2026-08-08 04:59:14 on dist-test-slave-gjw7"
server_key: "2cbf8a2552c3ddac233ebdc3b0944ee3"
server_key_iv: "9037202a03feecc2eec856320f78ff1d"
server_key_version: "encryptionkey@0"
I20260808 04:59:14.879912 22118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 04:59:14.880589 22143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:14.880728 22118 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 04:59:14.880797 22118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "a2c312b904a64137a4af23fbfc65e015"
format_stamp: "Formatted at 2026-08-08 04:59:14 on dist-test-slave-gjw7"
server_key: "2cbf8a2552c3ddac233ebdc3b0944ee3"
server_key_iv: "9037202a03feecc2eec856320f78ff1d"
server_key_version: "encryptionkey@0"
I20260808 04:59:14.880847 22118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:14.897845 22118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:14.899917 22118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:14.900025 22118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:14.900197 22118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:59:14.900486 22118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:59:14.900518 22118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:14.900547 22118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:59:14.900573 22118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:14.906271 22118 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:38301
I20260808 04:59:14.906337 22334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:38301 every 8 connection(s)
I20260808 04:59:14.906631 22118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 04:59:14 dist-test-slave-gjw7 krb5kdc[21722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165154, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:59:14.915577 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:14.907365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:52453 (local address 127.3.174.190:46659)
0808 04:59:14.907412 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:14.907414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:14.907612 (+ 198us) server_negotiation.cc:407] Connection header received
0808 04:59:14.908142 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:14.908143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:14.908155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:14.908207 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:14.909144 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.909704 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.910682 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.910873 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.910940 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:14.912937 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:14.912939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:14.912939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:59:14.912947 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:14.914314 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:14.914692 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:14.914693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:14.914694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:14.914749 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:14.914964 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:14.914965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:14.914965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:14.915074 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:14.915130 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:14.915390 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 04:59:14.915472 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 04:59:14.915740 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:14.907313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46659 (local address 127.3.174.130:52453)
0808 04:59:14.907493 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:14.907530 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:14.908029 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:14.908287 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:14.908291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:14.908844 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:14.909044 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:14.909049 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.909800 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.909801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:14.910594 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:14.910597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:14.910945 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:14.910946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:14.911075 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:14.911511 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:14.911519 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:14.912844 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:59:14.914402 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:59:14.914408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:59:14.914410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:59:14.914624 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:59:14.914805 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:59:14.914806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:59:14.914806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:59:14.914904 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:59:14.915132 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:59:14.915139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:59:14.915264 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 04:59:14.915354 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 04:59:14.915506 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260808 04:59:14.916280 22337 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46659
I20260808 04:59:14.916368 22337 heartbeater.cc:464] Registering TS with master...
I20260808 04:59:14.916543 22337 heartbeater.cc:511] Master 127.3.174.190:46659 requested a full tablet report, sending...
I20260808 04:59:14.916786 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 22118
I20260808 04:59:14.916893 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 04:59:14.916985 21810 ts_manager.cc:194] Registered new tserver with Master: a2c312b904a64137a4af23fbfc65e015 (127.3.174.130:38301)
I20260808 04:59:14.917114 3770 external_mini_cluster.cc:1428] Setting key 0695a00f78e9f786091497e99abe64c9
I20260808 04:59:14.917479 21810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:52453
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 04:59:14.942087 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46659
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 04:59:15.024009 22343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 04:59:15.024194 22343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 04:59:15.024215 22343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 04:59:15.025275 22343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 04:59:15.025333 22343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 04:59:15.025358 22343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 04:59:15.026759 22343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:46659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 04:59:15.026950 22343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 04:59:15.027120 22343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 04:59:15.031898 22349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:15.031873 22354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 04:59:15.031881 22350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 04:59:15.032131 22343 server_base.cc:1034] running on GCE node
I20260808 04:59:15.032322 22343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 04:59:15.032517 22343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 04:59:15.033654 22343 hybrid_clock.cc:648] HybridClock initialized: now 1786165155033631 us; error 35 us; skew 500 ppm
Aug 08 04:59:15 dist-test-slave-gjw7 krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165155, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 04:59:15.036223 22343 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 04:59:15.036648 22343 webserver.cc:460] Webserver started at http://127.3.174.131:41951/ using document root <none> and password file <none>
I20260808 04:59:15.036867 22343 fs_manager.cc:359] Metadata directory not provided
I20260808 04:59:15.036917 22343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 04:59:15.037041 22343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 04:59:15.038074 22343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "0a815a4de94348cb824089c6b2ca6907"
format_stamp: "Formatted at 2026-08-08 04:59:15 on dist-test-slave-gjw7"
server_key: "425fcd9f26b15e7dd8ea7cec61e4c25b"
server_key_iv: "6661b77a17722e2160063a900b88cebf"
server_key_version: "encryptionkey@0"
I20260808 04:59:15.038406 22343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "0a815a4de94348cb824089c6b2ca6907"
format_stamp: "Formatted at 2026-08-08 04:59:15 on dist-test-slave-gjw7"
server_key: "425fcd9f26b15e7dd8ea7cec61e4c25b"
server_key_iv: "6661b77a17722e2160063a900b88cebf"
server_key_version: "encryptionkey@0"
I20260808 04:59:15.039618 22343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 04:59:15.040254 22364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:15.040410 22343 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 04:59:15.040493 22343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "0a815a4de94348cb824089c6b2ca6907"
format_stamp: "Formatted at 2026-08-08 04:59:15 on dist-test-slave-gjw7"
server_key: "425fcd9f26b15e7dd8ea7cec61e4c25b"
server_key_iv: "6661b77a17722e2160063a900b88cebf"
server_key_version: "encryptionkey@0"
I20260808 04:59:15.040550 22343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 04:59:15.082329 22343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 04:59:15.084579 22343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 04:59:15.084728 22343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 04:59:15.084961 22343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 04:59:15.085283 22343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 04:59:15.085355 22343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:15.085388 22343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 04:59:15.085412 22343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 04:59:15.091269 22343 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:36337
I20260808 04:59:15.091396 22572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:36337 every 8 connection(s)
I20260808 04:59:15.091665 22343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 04:59:15.097152 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 22343
I20260808 04:59:15.097241 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 04:59:15.097429 3770 external_mini_cluster.cc:1428] Setting key 6875e7b50c9b7457f2c056c64bcee871
Aug 08 04:59:15 dist-test-slave-gjw7 krb5kdc[21722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165155, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 04:59:15.100632 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.092415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:44053 (local address 127.3.174.190:46659)
0808 04:59:15.092471 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:15.092473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:15.092691 (+ 218us) server_negotiation.cc:407] Connection header received
0808 04:59:15.093148 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:15.093150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:15.093160 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:15.093258 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:15.094222 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.094726 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.095763 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.095964 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.096021 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.098322 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:15.098324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:15.098324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:59:15.098332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:15.099568 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:15.099930 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:15.099931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:15.099931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:15.099979 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:15.100160 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:15.100160 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:15.100161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:15.100260 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:15.100305 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:15.100511 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 04:59:15.100569 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 04:59:15.100841 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.092376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46659 (local address 127.3.174.131:44053)
0808 04:59:15.092544 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:15.092587 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:15.093067 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:15.093355 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:15.093359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:15.093928 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 04:59:15.094125 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.094130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.094807 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.094808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.095681 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.095683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.096035 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.096036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.096156 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.096599 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 04:59:15.096608 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 04:59:15.098222 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 04:59:15.099638 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:59:15.099644 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:59:15.099646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:59:15.099874 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:59:15.100024 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 04:59:15.100025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 04:59:15.100025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 04:59:15.100117 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 04:59:15.100301 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 04:59:15.100305 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 04:59:15.100416 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 04:59:15.100494 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 04:59:15.100610 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260808 04:59:15.101239 22573 heartbeater.cc:347] Connected to a master server at 127.3.174.190:46659
I20260808 04:59:15.101347 22573 heartbeater.cc:464] Registering TS with master...
I20260808 04:59:15.101519 22573 heartbeater.cc:511] Master 127.3.174.190:46659 requested a full tablet report, sending...
I20260808 04:59:15.101909 21807 ts_manager.cc:194] Registered new tserver with Master: 0a815a4de94348cb824089c6b2ca6907 (127.3.174.131:36337)
I20260808 04:59:15.102293 21807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:44053
I20260808 04:59:15.109036 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 04:59:15.115605 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.110339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54542 (local address 127.3.174.190:46659)
0808 04:59:15.110377 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:15.110378 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:15.110403 (+ 25us) server_negotiation.cc:407] Connection header received
0808 04:59:15.110463 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:15.110464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:15.110471 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:15.110508 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 04:59:15.110993 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.111489 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.112161 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.112375 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.112427 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.113528 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 04:59:15.113529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 04:59:15.113529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 04:59:15.113535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 04:59:15.114686 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:15.114985 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:15.114986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:15.114987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:15.115031 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 04:59:15.115159 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 04:59:15.115160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 04:59:15.115160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 04:59:15.115321 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 04:59:15.115398 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:15.115488 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 04:59:15.115546 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 04:59:15.116954 21807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 04:59:15.118584 21807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 04:59:15.125139 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.120897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44360 (local address 127.3.174.131:36337)
0808 04:59:15.121167 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:15.121187 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:15.121198 (+ 11us) server_negotiation.cc:407] Connection header received
0808 04:59:15.121238 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:15.121240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:15.121257 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:15.121356 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:59:15.121636 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.122697 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.124067 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.124717 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.124779 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.124838 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:15.125011 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 04:59:15.125052 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":84,"threads_started":1}
I20260808 04:59:15.125160 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.120996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:38301 (local address 127.0.0.1:51000)
0808 04:59:15.121589 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:15.121598 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:15.121639 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:15.121841 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:15.121843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:15.121845 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:59:15.121901 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.121902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.123016 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.123017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.123887 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.123890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.124789 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.124790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.124899 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.124916 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 04:59:15.124989 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 04:59:15.125060 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":24,"thread_start_us":61,"threads_started":1}
I20260808 04:59:15.125160 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.120740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:36337 (local address 127.0.0.1:44360)
0808 04:59:15.120936 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:15.120976 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:15.121046 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:15.121417 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:15.121420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:15.121422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:59:15.121499 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.121500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.122795 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.122796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.123985 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.123988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.124789 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.124790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.124899 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.124916 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 04:59:15.124989 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 04:59:15.125060 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":38,"thread_start_us":149,"threads_started":1}
I20260808 04:59:15.125519 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.121125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.3.174.130:38301)
0808 04:59:15.121341 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:15.121346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:15.121706 (+ 360us) server_negotiation.cc:407] Connection header received
0808 04:59:15.121750 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:15.121752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:15.121767 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:15.121840 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:59:15.121965 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.122932 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.123961 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.124705 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.124780 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.124838 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:15.125400 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 04:59:15.125442 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":89,"threads_started":1}
I20260808 04:59:15.126307 22596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.120748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:46373 (local address 127.0.0.1:39290)
0808 04:59:15.121268 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:15.121278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:15.121340 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:15.121534 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:15.121535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:15.121537 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:59:15.121568 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.121569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.122552 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.122553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.123987 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.123988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.126106 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.126107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.126183 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.126187 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:59:15.126229 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 04:59:15.126258 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":7,"thread_start_us":98,"threads_started":1}
I20260808 04:59:15.126245 22224 tablet_service.cc:1467] Processing CreateTablet for tablet 16478041f3ed464aa6df921da8fc53e2 (DEFAULT_TABLE table=test-table [id=86d969d72d594b63a2aa3d5bd4d7a556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:59:15.126384 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.121099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39290 (local address 127.3.174.129:46373)
0808 04:59:15.121381 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:15.121384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:15.121394 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:59:15.121429 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:15.121431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:15.121443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:15.121538 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:59:15.121629 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.122459 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.124877 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.125898 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.126009 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.126072 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:15.126265 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 04:59:15.126309 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":80,"threads_started":1}
I20260808 04:59:15.126645 22224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16478041f3ed464aa6df921da8fc53e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:59:15.126951 22014 tablet_service.cc:1467] Processing CreateTablet for tablet 16478041f3ed464aa6df921da8fc53e2 (DEFAULT_TABLE table=test-table [id=86d969d72d594b63a2aa3d5bd4d7a556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:59:15.127252 22014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16478041f3ed464aa6df921da8fc53e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:59:15.129407 22605 tablet_bootstrap.cc:492] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: Bootstrap starting.
I20260808 04:59:15.129666 22459 tablet_service.cc:1467] Processing CreateTablet for tablet 16478041f3ed464aa6df921da8fc53e2 (DEFAULT_TABLE table=test-table [id=86d969d72d594b63a2aa3d5bd4d7a556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 04:59:15.130009 22459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16478041f3ed464aa6df921da8fc53e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 04:59:15.130244 22605 tablet_bootstrap.cc:654] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: Neither blocks nor log segments found. Creating new log.
I20260808 04:59:15.130777 22605 log.cc:824] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: Log is configured to *not* fsync() on all Append() calls
I20260808 04:59:15.131603 22605 tablet_bootstrap.cc:492] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: No bootstrap required, opened a new log
I20260808 04:59:15.131678 22605 ts_tablet_manager.cc:1397] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:59:15.133086 22610 tablet_bootstrap.cc:492] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: Bootstrap starting.
I20260808 04:59:15.133193 22607 tablet_bootstrap.cc:492] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21: Bootstrap starting.
I20260808 04:59:15.133760 22610 tablet_bootstrap.cc:654] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: Neither blocks nor log segments found. Creating new log.
I20260808 04:59:15.133635 22605 raft_consensus.cc:348] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } }
I20260808 04:59:15.133805 22605 raft_consensus.cc:374] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:59:15.133839 22605 raft_consensus.cc:729] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c312b904a64137a4af23fbfc65e015, State: Initialized, Role: FOLLOWER
I20260808 04:59:15.133927 22607 tablet_bootstrap.cc:654] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21: Neither blocks nor log segments found. Creating new log.
I20260808 04:59:15.133994 22605 consensus_queue.cc:260] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } }
I20260808 04:59:15.134323 22337 heartbeater.cc:503] Master 127.3.174.190:46659 was elected leader, sending a full tablet report...
I20260808 04:59:15.134347 22610 log.cc:824] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: Log is configured to *not* fsync() on all Append() calls
I20260808 04:59:15.134455 22607 log.cc:824] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21: Log is configured to *not* fsync() on all Append() calls
I20260808 04:59:15.134310 22605 ts_tablet_manager.cc:1428] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 04:59:15.135190 22607 tablet_bootstrap.cc:492] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21: No bootstrap required, opened a new log
I20260808 04:59:15.135185 22610 tablet_bootstrap.cc:492] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: No bootstrap required, opened a new log
I20260808 04:59:15.135257 22610 ts_tablet_manager.cc:1397] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:59:15.135257 22607 ts_tablet_manager.cc:1397] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 04:59:15.137120 22610 raft_consensus.cc:348] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } }
I20260808 04:59:15.137120 22607 raft_consensus.cc:348] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } }
I20260808 04:59:15.137280 22610 raft_consensus.cc:374] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:59:15.137280 22607 raft_consensus.cc:374] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 04:59:15.137346 22610 raft_consensus.cc:729] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a815a4de94348cb824089c6b2ca6907, State: Initialized, Role: FOLLOWER
I20260808 04:59:15.137346 22607 raft_consensus.cc:729] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ba0efaaf024444941486579832bc21, State: Initialized, Role: FOLLOWER
I20260808 04:59:15.137462 22610 consensus_queue.cc:260] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } }
I20260808 04:59:15.137462 22607 consensus_queue.cc:260] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } }
I20260808 04:59:15.137734 22610 ts_tablet_manager.cc:1428] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 04:59:15.137734 22607 ts_tablet_manager.cc:1428] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 04:59:15.137856 22113 heartbeater.cc:503] Master 127.3.174.190:46659 was elected leader, sending a full tablet report...
I20260808 04:59:15.137768 22573 heartbeater.cc:503] Master 127.3.174.190:46659 was elected leader, sending a full tablet report...
W20260808 04:59:15.157174 22338 tablet.cc:2369] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:59:15.250960 22114 tablet.cc:2369] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 04:59:15.342242 22575 tablet.cc:2369] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 04:59:15.363423 22618 raft_consensus.cc:482] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 04:59:15.363556 22618 raft_consensus.cc:504] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } }
I20260808 04:59:15.363933 22618 leader_election.cc:288] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93ba0efaaf024444941486579832bc21 (127.3.174.129:46373), a2c312b904a64137a4af23fbfc65e015 (127.3.174.130:38301)
I20260808 04:59:15.366850 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.364178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:46373 (local address 127.3.174.131:43965)
0808 04:59:15.364236 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:15.364248 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:15.364301 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:15.364436 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:15.364437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:15.364438 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:59:15.364512 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.364514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.365323 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.365325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.366049 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.366051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.366593 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.366594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.366674 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.366677 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 04:59:15.366732 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 04:59:15.366774 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 04:59:15.366850 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.364262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:43965 (local address 127.3.174.129:46373)
0808 04:59:15.364331 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:15.364334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:15.364344 (+ 10us) server_negotiation.cc:407] Connection header received
0808 04:59:15.364372 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:15.364373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:15.364383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:15.364432 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:59:15.364610 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.365215 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.366146 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.366507 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.366587 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.366597 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:15.366743 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 04:59:15.366781 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 04:59:15.366981 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.364304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:38301 (local address 127.3.174.131:36335)
0808 04:59:15.364501 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:15.364512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 04:59:15.364571 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 04:59:15.364743 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 04:59:15.364744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 04:59:15.364745 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 04:59:15.364818 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.364820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.365454 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.365455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.366186 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 04:59:15.366188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.366737 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.366738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 04:59:15.366809 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.366813 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 04:59:15.366864 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 04:59:15.366901 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":1,"thread_start_us":125,"threads_started":1}
I20260808 04:59:15.366974 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.364591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:36335 (local address 127.3.174.130:38301)
0808 04:59:15.364638 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:15.364640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:15.364647 (+ 7us) server_negotiation.cc:407] Connection header received
0808 04:59:15.364671 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:15.364672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:15.364680 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:15.364719 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 04:59:15.364891 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.365380 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.366274 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.366656 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.366718 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.366727 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:15.366878 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 04:59:15.366915 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 04:59:15.367274 22042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16478041f3ed464aa6df921da8fc53e2" candidate_uuid: "0a815a4de94348cb824089c6b2ca6907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ba0efaaf024444941486579832bc21" is_pre_election: true
I20260808 04:59:15.367273 22267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16478041f3ed464aa6df921da8fc53e2" candidate_uuid: "0a815a4de94348cb824089c6b2ca6907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c312b904a64137a4af23fbfc65e015" is_pre_election: true
I20260808 04:59:15.367410 22267 raft_consensus.cc:2454] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a815a4de94348cb824089c6b2ca6907 in term 0.
I20260808 04:59:15.367424 22042 raft_consensus.cc:2454] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a815a4de94348cb824089c6b2ca6907 in term 0.
I20260808 04:59:15.367651 22381 leader_election.cc:302] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a815a4de94348cb824089c6b2ca6907, 93ba0efaaf024444941486579832bc21; no voters:
I20260808 04:59:15.367795 22618 raft_consensus.cc:2781] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 04:59:15.367857 22618 raft_consensus.cc:482] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 04:59:15.367880 22618 raft_consensus.cc:3037] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:59:15.368607 22618 raft_consensus.cc:504] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } }
I20260808 04:59:15.368726 22618 leader_election.cc:288] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [CANDIDATE]: Term 1 election: Requested vote from peers 93ba0efaaf024444941486579832bc21 (127.3.174.129:46373), a2c312b904a64137a4af23fbfc65e015 (127.3.174.130:38301)
I20260808 04:59:15.368922 22267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16478041f3ed464aa6df921da8fc53e2" candidate_uuid: "0a815a4de94348cb824089c6b2ca6907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c312b904a64137a4af23fbfc65e015"
I20260808 04:59:15.368932 22042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16478041f3ed464aa6df921da8fc53e2" candidate_uuid: "0a815a4de94348cb824089c6b2ca6907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ba0efaaf024444941486579832bc21"
I20260808 04:59:15.368999 22267 raft_consensus.cc:3037] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:59:15.369002 22042 raft_consensus.cc:3037] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 04:59:15.369731 22267 raft_consensus.cc:2454] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a815a4de94348cb824089c6b2ca6907 in term 1.
I20260808 04:59:15.369846 22042 raft_consensus.cc:2454] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a815a4de94348cb824089c6b2ca6907 in term 1.
I20260808 04:59:15.369926 22381 leader_election.cc:302] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a815a4de94348cb824089c6b2ca6907, a2c312b904a64137a4af23fbfc65e015; no voters:
I20260808 04:59:15.370066 22618 raft_consensus.cc:2781] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 04:59:15.370237 22618 raft_consensus.cc:686] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 1 LEADER]: Becoming Leader. State: Replica: 0a815a4de94348cb824089c6b2ca6907, State: Running, Role: LEADER
I20260808 04:59:15.370344 22618 consensus_queue.cc:237] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } }
I20260808 04:59:15.371249 21807 catalog_manager.cc:5495] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a815a4de94348cb824089c6b2ca6907 (127.3.174.131). New cstate: current_term: 1 leader_uuid: "0a815a4de94348cb824089c6b2ca6907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a815a4de94348cb824089c6b2ca6907" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 36337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 } health_report { overall_health: UNKNOWN } } }
I20260808 04:59:15.406343 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.403972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.3.174.131:36337)
0808 04:59:15.404030 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 04:59:15.404032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 04:59:15.404040 (+ 8us) server_negotiation.cc:407] Connection header received
0808 04:59:15.404074 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 04:59:15.404075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 04:59:15.404092 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 04:59:15.404154 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 04:59:15.404282 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.404761 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.405588 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 04:59:15.405827 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 04:59:15.405892 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 04:59:15.406028 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 04:59:15.406132 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 04:59:15.406188 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 04:59:15.406222 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 04:59:15.406242 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 04:59:15.408136 22267 raft_consensus.cc:1261] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [term 1 FOLLOWER]: Refusing update from remote peer 0a815a4de94348cb824089c6b2ca6907: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:59:15.408149 22042 raft_consensus.cc:1261] T 16478041f3ed464aa6df921da8fc53e2 P 93ba0efaaf024444941486579832bc21 [term 1 FOLLOWER]: Refusing update from remote peer 0a815a4de94348cb824089c6b2ca6907: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 04:59:15.408504 22618 consensus_queue.cc:1035] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2c312b904a64137a4af23fbfc65e015" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:59:15.408663 22624 consensus_queue.cc:1035] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93ba0efaaf024444941486579832bc21" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 46373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 04:59:15.413354 21807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:59:15.413447 21807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 04:59:15.414124 21807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ab71fe0cf6f4de892f6aae0172ead62: Sending DeleteTablet for 3 replicas of tablet 16478041f3ed464aa6df921da8fc53e2
I20260808 04:59:15.414433 22459 tablet_service.cc:1514] Processing DeleteTablet for tablet 16478041f3ed464aa6df921da8fc53e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:59:15 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:44360
I20260808 04:59:15.414462 22014 tablet_service.cc:1514] Processing DeleteTablet for tablet 16478041f3ed464aa6df921da8fc53e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:59:15 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:39290
I20260808 04:59:15.414638 22645 tablet_replica.cc:330] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: stopping tablet replica
I20260808 04:59:15.414681 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 21910
I20260808 04:59:15.414701 22224 tablet_service.cc:1514] Processing DeleteTablet for tablet 16478041f3ed464aa6df921da8fc53e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 04:59:15 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:51000
I20260808 04:59:15.414731 22645 raft_consensus.cc:2229] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 1 LEADER]: Raft consensus shutting down.
I20260808 04:59:15.414817 22645 raft_consensus.cc:2258] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:59:15.430181 22645 ts_tablet_manager.cc:1905] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:59:15.432133 22645 ts_tablet_manager.cc:1918] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:59:15.432298 22645 log.cc:1196] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/16478041f3ed464aa6df921da8fc53e2
I20260808 04:59:15.432675 22645 ts_tablet_manager.cc:1939] T 16478041f3ed464aa6df921da8fc53e2 P 0a815a4de94348cb824089c6b2ca6907: Deleting consensus metadata
I20260808 04:59:15.433218 21781 catalog_manager.cc:4841] TS 0a815a4de94348cb824089c6b2ca6907 (127.3.174.131:36337): tablet 16478041f3ed464aa6df921da8fc53e2 (table test-table [id=86d969d72d594b63a2aa3d5bd4d7a556]) successfully deleted
I20260808 04:59:15.433646 22647 tablet_replica.cc:330] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: stopping tablet replica
I20260808 04:59:15.433786 22647 raft_consensus.cc:2229] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 04:59:15.433916 22647 raft_consensus.cc:2258] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 04:59:15.434401 22647 ts_tablet_manager.cc:1905] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 04:59:15.435943 22647 ts_tablet_manager.cc:1918] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 04:59:15.436072 22647 log.cc:1196] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/16478041f3ed464aa6df921da8fc53e2
I20260808 04:59:15.436437 22647 ts_tablet_manager.cc:1939] T 16478041f3ed464aa6df921da8fc53e2 P a2c312b904a64137a4af23fbfc65e015: Deleting consensus metadata
I20260808 04:59:15.436937 21784 catalog_manager.cc:4841] TS a2c312b904a64137a4af23fbfc65e015 (127.3.174.130:38301): tablet 16478041f3ed464aa6df921da8fc53e2 (table test-table [id=86d969d72d594b63a2aa3d5bd4d7a556]) successfully deleted
W20260808 04:59:15.438848 21784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:46373 (error 108)
I20260808 04:59:15.439419 22596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 04:59:15.439285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:46373 (local address 127.0.0.1:39306)
0808 04:59:15.439338 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 04:59:15.439381 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:46373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 04:59:15.439446 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 22118
W20260808 04:59:15.448377 21784 catalog_manager.cc:4584] TS 93ba0efaaf024444941486579832bc21 (127.3.174.129:46373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16478041f3ed464aa6df921da8fc53e2: Network error: Client connection negotiation failed: client connection to 127.3.174.129:46373: connect: Connection refused (error 111)
I20260808 04:59:15.469336 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 22343
I20260808 04:59:15.485116 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 21743
2026-08-08T04:59:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2324 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 04:59:15 dist-test-slave-gjw7 krb5kdc[22654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 04:59:15 dist-test-slave-gjw7 krb5kdc[22654](info): set up 2 sockets
Aug 08 04:59:15 dist-test-slave-gjw7 krb5kdc[22654](info): commencing operation
krb5kdc: starting...
W20260808 04:59:16.674124 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 04:59:16 dist-test-slave-gjw7 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T04:59:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T04:59:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.174.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.174.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/rangeradmin_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/rangeradmin_127.3.174.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.174.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.174.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/rangerlookup_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/rangerlookup_127.3.174.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.174.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.174.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/HTTP_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/HTTP_127.3.174.148@KRBTEST.COM.keytab.
I20260808 04:59:16.831257 3770 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskihVxiI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/postgres -l logfile start
2026-08-08 04:59:19.161 UTC [22704] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 04:59:19.161 UTC [22704] LOG: listening on IPv4 address "127.3.174.148", port 51161
2026-08-08 04:59:19.165 UTC [22704] LOG: listening on Unix socket "/tmp/.s.PGSQL.51161"
2026-08-08 04:59:19.186 UTC [22707] LOG: database system was shut down at 2026-08-08 04:59:17 UTC
2026-08-08 04:59:19.189 UTC [22704] LOG: database system is ready to accept connections
I20260808 04:59:20.297821 3770 mini_postgres.cc:96] Postgres bound to 51161
2026-08-08 04:59:20.305 UTC [22721] FATAL: database "slave" does not exist
127.3.174.148:51161 - accepting connections
I20260808 04:59:20.305960 3770 mini_ranger.cc:161] Starting Ranger...
I20260808 04:59:20.320881 3770 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 04:59:20.778095 3770 mini_ranger.cc:87] Created ranger Postgres database
I20260808 04:59:20.778177 3770 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-admin
2026-08-08 04:59:21,092 [I] DB FLAVOR :POSTGRES
2026-08-08 04:59:21,092 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 04:59:21,092 [I] Checking connection..
2026-08-08 04:59:21,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 04:59:21,364 [I] Checking connection passed.
2026-08-08 04:59:21,364 [I] --------- Verifying version history table ---------
2026-08-08 04:59:21,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 04:59:21,621 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 04:59:21,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 04:59:21,862 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 04:59:21,862 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 04:59:21,862 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 04:59:22.115 UTC [22824] WARNING: there is no transaction in progress
2026-08-08 04:59:22,123 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 04:59:22,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 04:59:22,370 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 04:59:22,370 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 04:59:22,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 04:59:22,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 04:59:22,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjw7.c.gcp-upstream.internal','N') ;"
2026-08-08 04:59:23,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 04:59:23,404 [I] Table x_portal_user does not exist in database ranger
2026-08-08 04:59:23,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 04:59:23,643 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 04:59:23,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 04:59:23,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 04:59:24,138 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 04:59:24,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 04:59:25.058 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.074 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.090 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.104 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.119 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.136 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.188 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.195 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.209 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.219 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.229 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.243 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.252 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.258 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.264 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25.270 UTC [23011] WARNING: there is no transaction in progress
2026-08-08 04:59:25,883 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 04:59:25,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 04:59:26,136 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 04:59:26,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 04:59:26,366 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 04:59:26,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 04:59:26,623 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 04:59:26,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 04:59:26,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 04:59:27,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjw7.c.gcp-upstream.internal';"
2026-08-08 04:59:27,380 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 04:59:27,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 04:59:27,629 [I] DB_PATCHES have already been applied
I20260808 04:59:27.634543 3770 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-admin:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskihVxiI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 04:59:27.634626 3770 mini_ranger.cc:193] Using host: 127.3.174.148
I20260808 04:59:27.638480 3770 mini_ranger.cc:227] Ranger admin URL: http://127.3.174.148:49757
Aug 08, 2026 4:59:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:59:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 4:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 4:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 4:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 4:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 04:59:28.922034 3770 mini_ranger.cc:160] Time spent starting Ranger: real 8.616s user 0.002s sys 0.008s
Aug 08, 2026 4:59:49 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 4:59:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 4:59:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 4:59:49 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 4:59:50 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 04:59:51.341784 3770 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.174.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.174.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/rangerkms_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/rangerkms_127.3.174.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.174.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.174.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.174.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/HTTP_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.174.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/HTTP_127.3.174.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 04:59:51 dist-test-slave-gjw7 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165191, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 04:59:51.429004 3770 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 04:59:51.441102 3770 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 04:59:52.261622 3770 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 04:59:52.261698 3770 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-kms
I20260808 04:59:52.261709 3770 mini_ranger_kms.cc:227] Using postgres at 127.3.174.148:51161
2026-08-08 04:59:53,023 [I] DB FLAVOR :POSTGRES
2026-08-08 04:59:53,024 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 04:59:53,024 [I] Checking connection
2026-08-08 04:59:53,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 04:59:53,467 [I] connection success
2026-08-08 04:59:53,468 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 04:59:53,468 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 04:59:53,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 04:59:53,706 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 04:59:53,706 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 04:59:53,706 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 04:59:53,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:51161/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 04:59:53,975 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 04:59:54.428517 3770 mini_ranger_kms.cc:310] Created kms service
I20260808 04:59:54.674043 3770 mini_ranger_kms.cc:326] Created kudu user
I20260808 04:59:54.720716 3770 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 04:59:54 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165191, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 04:59:55.128068 3770 mini_ranger_kms.cc:384] Added ranger policy
I20260808 04:59:55.128197 3770 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-kms:/tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 04:59:55.128223 3770 mini_ranger_kms.cc:240] Using host: 127.3.174.148
I20260808 04:59:55.132830 3770 mini_ranger_kms.cc:276] Ranger KMS PID: 23278
I20260808 04:59:55.132921 3770 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.174.148:37699
Aug 08, 2026 4:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 4:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 4:59:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 4:59:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 04:59:57.875170 3770 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.446s user 0.006s sys 0.727s
I20260808 04:59:57.875397 3770 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 04:59:57.875490 3770 mini_ranger_kms.cc:401] 127.3.174.148:37699/kms/v1/keys
Aug 08 04:59:57 dist-test-slave-gjw7 krb5kdc[22654](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786165197, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.174.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 04:59:58 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165197, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.174.148@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
Aug 08 04:59:58 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165197, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.174.148@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
Aug 08 05:00:01 dist-test-slave-gjw7 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:00:01.510649 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35155
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:34159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:35155
--ranger_config_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:37699/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:01.630774 23367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:01.631003 23367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:01.631028 23367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:01.632282 23367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:00:01.632315 23367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:01.632328 23367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:01.632340 23367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:00:01.632351 23367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:00:01.634235 23367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:34159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.174.148:37699/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:35155
--ranger_config_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:01.634466 23367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:01.634656 23367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:01.641193 23375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:01.641270 23376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:01.641310 23367 server_base.cc:1034] running on GCE node
W20260808 05:00:01.641320 23380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:01.641764 23367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:01.642043 23367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:01.643195 23367 hybrid_clock.cc:648] HybridClock initialized: now 1786165201643173 us; error 41 us; skew 500 ppm
Aug 08 05:00:01 dist-test-slave-gjw7 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165201, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:01.649484 23367 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:00:01.649945 23367 webserver.cc:460] Webserver started at http://127.3.174.190:43937/ using document root <none> and password file <none>
I20260808 05:00:01.650168 23367 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:01.650213 23367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:01.650339 23367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 05:00:01 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165201, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:01.703454 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "2ec88846ecf04188a6f66f3790c586b6"
format_stamp: "Formatted at 2026-08-08 05:00:01 on dist-test-slave-gjw7"
server_key: "8bdc8582d0fab88a0e7391cb17407bb6"
server_key_iv: "9101bfa2698f91f16b4c34b6496869d3"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:01.703962 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "2ec88846ecf04188a6f66f3790c586b6"
format_stamp: "Formatted at 2026-08-08 05:00:01 on dist-test-slave-gjw7"
server_key: "8bdc8582d0fab88a0e7391cb17407bb6"
server_key_iv: "9101bfa2698f91f16b4c34b6496869d3"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:01.705956 23367 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260808 05:00:01.721457 23388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:01.721776 23367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:00:01.721887 23367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "2ec88846ecf04188a6f66f3790c586b6"
format_stamp: "Formatted at 2026-08-08 05:00:01 on dist-test-slave-gjw7"
server_key: "8bdc8582d0fab88a0e7391cb17407bb6"
server_key_iv: "9101bfa2698f91f16b4c34b6496869d3"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:01.721987 23367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:01.763366 23367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:01.765570 23367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:01.765825 23367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:01.776525 23367 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:35155
I20260808 05:00:01.776612 23464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:35155 every 8 connection(s)
I20260808 05:00:01.777005 23367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:00:01.777891 23465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:01.779896 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 23367
I20260808 05:00:01.779984 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:00:01.780154 3770 external_mini_cluster.cc:1428] Setting key a1f6afa8fad092a02459bbe13d6a519c
I20260808 05:00:01.781337 23465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6: Bootstrap starting.
I20260808 05:00:01.782691 23465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:01.783396 23465 log.cc:824] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:01.784212 23465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6: No bootstrap required, opened a new log
I20260808 05:00:01.787835 23465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec88846ecf04188a6f66f3790c586b6" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35155 } }
I20260808 05:00:01.788020 23465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:01.788057 23465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec88846ecf04188a6f66f3790c586b6, State: Initialized, Role: FOLLOWER
Aug 08 05:00:01 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:01.788192 23465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec88846ecf04188a6f66f3790c586b6" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35155 } }
I20260808 05:00:01.788265 23465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:00:01.788303 23465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:00:01.788354 23465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:01.789268 23465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec88846ecf04188a6f66f3790c586b6" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35155 } }
I20260808 05:00:01.789405 23465 leader_election.cc:302] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ec88846ecf04188a6f66f3790c586b6; no voters:
I20260808 05:00:01.789578 23465 leader_election.cc:288] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:00:01.789767 23473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:00:01.790081 23465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:00:01.790081 23473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ec88846ecf04188a6f66f3790c586b6, State: Running, Role: LEADER
I20260808 05:00:01.790337 23473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec88846ecf04188a6f66f3790c586b6" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35155 } }
I20260808 05:00:01.790863 23474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ec88846ecf04188a6f66f3790c586b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec88846ecf04188a6f66f3790c586b6" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35155 } } }
I20260808 05:00:01.790961 23474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:01.791638 23476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ec88846ecf04188a6f66f3790c586b6. Latest consensus state: current_term: 1 leader_uuid: "2ec88846ecf04188a6f66f3790c586b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec88846ecf04188a6f66f3790c586b6" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35155 } } }
I20260808 05:00:01.791651 23482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:00:01.791749 23476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:01.792897 23482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:00:01.793313 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:01.780719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.3.174.190:35155)
0808 05:00:01.781033 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:01.781043 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:01.781114 (+ 71us) server_negotiation.cc:407] Connection header received
0808 05:00:01.782013 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:01.782025 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:01.782095 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:01.782281 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:01.782985 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:01.784412 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:01.785155 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:01.785981 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:01.786105 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:01.788753 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:01.788758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:01.788758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:01.788771 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:01.791034 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:01.791464 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:01.791468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:01.791468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:01.791574 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:01.791812 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:01.791812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:01.791813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:01.792132 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:01.792266 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:01.792867 (+ 601us) server_negotiation.cc:299] Negotiation successful
0808 05:00:01.792998 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":208,"threads_started":1}
I20260808 05:00:01.793569 23465 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 05:00:01.801205 23482 catalog_manager.cc:1347] Generated new cluster ID: cb208f9a4c8840d8971c4673c531e8df
I20260808 05:00:01.801282 23482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:00:01.820000 23482 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:00:01.820772 23482 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:00:01.827661 23482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6: Generated new TSK 0
I20260808 05:00:01.827915 23482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 05:00:02.764400 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:37699/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35155
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:34159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:02.878250 23540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:02.878595 23540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:02.878664 23540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:02.879887 23540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:02.879933 23540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:02.879973 23540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:00:02.881693 23540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:34159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.174.148:37699/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:02.881940 23540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:02.882179 23540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:02.888844 23547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:02.888916 23546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:02.889014 23549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:02.889333 23540 server_base.cc:1034] running on GCE node
I20260808 05:00:02.889545 23540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:02.889792 23540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:02.891010 23540 hybrid_clock.cc:648] HybridClock initialized: now 1786165202890990 us; error 33 us; skew 500 ppm
Aug 08 05:00:02 dist-test-slave-gjw7 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165202, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:02.894284 23540 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:00:02.894742 23540 webserver.cc:460] Webserver started at http://127.3.174.129:35485/ using document root <none> and password file <none>
I20260808 05:00:02.894963 23540 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:02.895011 23540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:02.895174 23540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 05:00:02 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165202, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:02.909937 23540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "109c5c4d42bb46329bfc697a403eaa8c"
format_stamp: "Formatted at 2026-08-08 05:00:02 on dist-test-slave-gjw7"
server_key: "20d85580fbfd9b3dae432da619d67cd6"
server_key_iv: "93ca7b4e9fc818a876e57dba3c9600f8"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:02.910537 23540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "109c5c4d42bb46329bfc697a403eaa8c"
format_stamp: "Formatted at 2026-08-08 05:00:02 on dist-test-slave-gjw7"
server_key: "20d85580fbfd9b3dae432da619d67cd6"
server_key_iv: "93ca7b4e9fc818a876e57dba3c9600f8"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:02.912091 23540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:00:02.925081 23562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:02.925316 23540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 05:00:02.925428 23540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "109c5c4d42bb46329bfc697a403eaa8c"
format_stamp: "Formatted at 2026-08-08 05:00:02 on dist-test-slave-gjw7"
server_key: "20d85580fbfd9b3dae432da619d67cd6"
server_key_iv: "93ca7b4e9fc818a876e57dba3c9600f8"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:02.925527 23540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:02.939570 23540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:02.942009 23540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:02.942277 23540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:02.942679 23540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:02.943163 23540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:02.943257 23540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:02.943361 23540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:02.943437 23540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:02.956984 23540 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:36361
I20260808 05:00:02.957633 23540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:00:02.964010 23764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:36361 every 8 connection(s)
I20260808 05:00:02.965639 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 23540
I20260808 05:00:02.965749 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:00:02.965955 3770 external_mini_cluster.cc:1428] Setting key 0af27faad1d7b1178469078c33fc56fc
Aug 08 05:00:02 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165202, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:02.976912 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:02.964438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:39381 (local address 127.3.174.190:35155)
0808 05:00:02.964600 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:02.964603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:02.967069 (+ 2466us) server_negotiation.cc:407] Connection header received
0808 05:00:02.967949 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:02.967950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:02.967964 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:02.968010 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:02.969169 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:02.969986 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:02.971472 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:02.971683 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:02.971739 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:02.974180 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:02.974182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:02.974182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:02.974190 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:02.975570 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:02.975966 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:02.975967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:02.975968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:02.976024 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:02.976254 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:02.976255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:02.976256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:02.976371 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:02.976428 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:02.976709 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 05:00:02.976793 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":57,"threads_started":1}
I20260808 05:00:02.977054 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:02.964285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35155 (local address 127.3.174.129:39381)
0808 05:00:02.966894 (+ 2609us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:02.966945 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:02.967807 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:02.968230 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:02.968235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:02.968642 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:02.969057 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:02.969065 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:02.970093 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:02.970094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:02.971379 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:02.971383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:02.971761 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:02.971761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:02.971939 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:02.972548 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:02.972557 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:02.974089 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:02.975651 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:02.975659 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:02.975662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:02.975882 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:02.976082 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:02.976083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:02.976083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:02.976195 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:02.976433 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:02.976440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:02.976583 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 05:00:02.976677 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 05:00:02.976823 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2548,"thread_start_us":944,"threads_started":1}
I20260808 05:00:02.977762 23766 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35155
I20260808 05:00:02.977869 23766 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:02.978060 23766 heartbeater.cc:511] Master 127.3.174.190:35155 requested a full tablet report, sending...
I20260808 05:00:02.978615 23411 ts_manager.cc:194] Registered new tserver with Master: 109c5c4d42bb46329bfc697a403eaa8c (127.3.174.129:36361)
I20260808 05:00:02.979223 23411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:39381
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 05:00:03.002568 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:37699/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35155
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:34159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:03.136667 23777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:03.136920 23777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:03.136957 23777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:03.138257 23777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:03.138305 23777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:03.138340 23777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:00:03.140213 23777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:34159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.174.148:37699/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:03.140466 23777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:03.140726 23777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:03.147014 23791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:03.147266 23788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:03.147290 23789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:03.147712 23777 server_base.cc:1034] running on GCE node
I20260808 05:00:03.147933 23777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:03.148177 23777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:03.149361 23777 hybrid_clock.cc:648] HybridClock initialized: now 1786165203149345 us; error 25 us; skew 500 ppm
Aug 08 05:00:03 dist-test-slave-gjw7 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:03.152644 23777 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:00:03.153164 23777 webserver.cc:460] Webserver started at http://127.3.174.130:45485/ using document root <none> and password file <none>
I20260808 05:00:03.153461 23777 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:03.153513 23777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:03.153649 23777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 05:00:03 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:03.171087 23777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "644de1260b5944fd87078852b5cbae6b"
format_stamp: "Formatted at 2026-08-08 05:00:03 on dist-test-slave-gjw7"
server_key: "fdab347dbf788369ab0c27316f016ab1"
server_key_iv: "443fb4900d7626b2ca5478eeb7535513"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:03.171599 23777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "644de1260b5944fd87078852b5cbae6b"
format_stamp: "Formatted at 2026-08-08 05:00:03 on dist-test-slave-gjw7"
server_key: "fdab347dbf788369ab0c27316f016ab1"
server_key_iv: "443fb4900d7626b2ca5478eeb7535513"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:03.177012 23777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
Aug 08 05:00:03 dist-test-slave-gjw7 krb5kdc[22654](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786165203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:03.205510 23803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:03.205734 23777 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.001s sys 0.001s
I20260808 05:00:03.205830 23777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "644de1260b5944fd87078852b5cbae6b"
format_stamp: "Formatted at 2026-08-08 05:00:03 on dist-test-slave-gjw7"
server_key: "fdab347dbf788369ab0c27316f016ab1"
server_key_iv: "443fb4900d7626b2ca5478eeb7535513"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:03.205943 23777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:03.246552 23777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:03.248788 23777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:03.248950 23777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:03.249208 23777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:03.249650 23777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:03.249694 23777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:03.249727 23777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:03.249758 23777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:03.267205 23777 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:36225
I20260808 05:00:03.267619 23777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 05:00:03.269907 23988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:36225 every 8 connection(s)
I20260808 05:00:03.274638 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 23777
I20260808 05:00:03.274755 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:00:03.274961 3770 external_mini_cluster.cc:1428] Setting key d7811e579552a94381260d1b452b409b
Aug 08 05:00:03 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:03.284260 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.270749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:60047 (local address 127.3.174.190:35155)
0808 05:00:03.270826 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:03.270829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:03.271193 (+ 364us) server_negotiation.cc:407] Connection header received
0808 05:00:03.271819 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:03.271820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:03.271837 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:03.271926 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:03.272768 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.273575 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.274838 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.275120 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.275167 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.280801 (+ 5634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:03.280803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:03.280803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:03.280812 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:03.282473 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:03.283040 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:03.283041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:03.283041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:03.283109 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:03.283425 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:03.283426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:03.283427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:03.283571 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:03.283617 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:03.284037 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 05:00:03.284136 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 05:00:03.284844 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.270685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35155 (local address 127.3.174.130:60047)
0808 05:00:03.271008 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:03.271056 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:03.271697 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:03.271987 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:03.271990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:03.272346 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:03.272655 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.272662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.273670 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.273670 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.274728 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.274731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.275253 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.275254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.275459 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.278719 (+ 3260us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:03.278732 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:03.280679 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:03.282577 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:03.282587 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:03.282590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:03.282940 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:03.283183 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:03.283183 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:03.283183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:03.283344 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:03.283663 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:03.283670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:03.283866 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 05:00:03.284373 (+ 507us) client_negotiation.cc:240] Negotiation successful
0808 05:00:03.284556 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":191,"threads_started":1}
I20260808 05:00:03.285492 23989 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35155
I20260808 05:00:03.285604 23989 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:03.285849 23989 heartbeater.cc:511] Master 127.3.174.190:35155 requested a full tablet report, sending...
I20260808 05:00:03.286304 23411 ts_manager.cc:194] Registered new tserver with Master: 644de1260b5944fd87078852b5cbae6b (127.3.174.130:36225)
I20260808 05:00:03.286832 23411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:60047
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 05:00:03.312337 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:37699/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35155
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:34159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:03.473896 23996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:03.474231 23996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:03.474282 23996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:03.476114 23996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:03.476186 23996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:03.476234 23996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:00:03.478701 23996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:34159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.174.148:37699/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:03.478973 23996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:03.479310 23996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:03.486207 24013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:03.486197 24005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:03.486564 24008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:03.487543 23996 server_base.cc:1034] running on GCE node
I20260808 05:00:03.487743 23996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:03.487983 23996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:03.489130 23996 hybrid_clock.cc:648] HybridClock initialized: now 1786165203489105 us; error 48 us; skew 500 ppm
Aug 08 05:00:03 dist-test-slave-gjw7 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:03.493994 23996 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:00:03.494604 23996 webserver.cc:460] Webserver started at http://127.3.174.131:34297/ using document root <none> and password file <none>
I20260808 05:00:03.494844 23996 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:03.494894 23996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:03.495038 23996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 05:00:03 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:03.509483 23996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "47357dc2a79d456fbe596826e4d404ef"
format_stamp: "Formatted at 2026-08-08 05:00:03 on dist-test-slave-gjw7"
server_key: "e006a429b2aab9ff9b5ba4d5d20b1386"
server_key_iv: "9d2706422b4363335f2c795344d513a0"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:03.510074 23996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "47357dc2a79d456fbe596826e4d404ef"
format_stamp: "Formatted at 2026-08-08 05:00:03 on dist-test-slave-gjw7"
server_key: "e006a429b2aab9ff9b5ba4d5d20b1386"
server_key_iv: "9d2706422b4363335f2c795344d513a0"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:03.512995 23996 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 05:00:03.526026 24025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:03.526856 23996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 05:00:03.526983 23996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "47357dc2a79d456fbe596826e4d404ef"
format_stamp: "Formatted at 2026-08-08 05:00:03 on dist-test-slave-gjw7"
server_key: "e006a429b2aab9ff9b5ba4d5d20b1386"
server_key_iv: "9d2706422b4363335f2c795344d513a0"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:03.527098 23996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:03.553201 23996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:03.555877 23996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:03.556083 23996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:03.556546 23996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:03.557009 23996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:03.557062 23996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:03.557094 23996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:03.557111 23996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:03.572484 23996 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:45361
I20260808 05:00:03.573184 23996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:00:03.575471 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 23996
I20260808 05:00:03.575580 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 05:00:03.575795 3770 external_mini_cluster.cc:1428] Setting key ca2c8e03988093d5b1718efff82139ac
I20260808 05:00:03.577476 24216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:45361 every 8 connection(s)
Aug 08 05:00:03 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:03.598913 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.574177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:47415 (local address 127.3.174.190:35155)
0808 05:00:03.574235 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:03.574238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:03.576340 (+ 2102us) server_negotiation.cc:407] Connection header received
0808 05:00:03.577530 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:03.577532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:03.577544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:03.577572 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:03.578618 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.579401 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.580941 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.581240 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.581334 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.584333 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:03.584335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:03.584335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:03.584343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:03.586039 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:03.586500 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:03.586502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:03.586502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:03.586572 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:03.586866 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:03.586867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:03.586867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:03.597544 (+ 10677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:03.597606 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:03.598670 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0808 05:00:03.598803 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 05:00:03.599571 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.574168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35155 (local address 127.3.174.131:47415)
0808 05:00:03.576134 (+ 1966us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:03.576202 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:03.577408 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:03.577816 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:03.577821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:03.578153 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:03.578496 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.578505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.579512 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.579513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.580839 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.580842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.581361 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.581362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.581537 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.582196 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:03.582207 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:03.584224 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:03.586126 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:03.586135 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:03.586138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:03.586434 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:03.586633 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:03.586634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:03.586634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:03.586783 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:03.598236 (+ 11453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:03.598251 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:03.598482 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 05:00:03.599073 (+ 591us) client_negotiation.cc:240] Negotiation successful
0808 05:00:03.599284 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1875,"thread_start_us":212,"threads_started":1}
I20260808 05:00:03.600080 24218 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35155
I20260808 05:00:03.600199 24218 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:03.600435 24218 heartbeater.cc:511] Master 127.3.174.190:35155 requested a full tablet report, sending...
I20260808 05:00:03.600864 23411 ts_manager.cc:194] Registered new tserver with Master: 47357dc2a79d456fbe596826e4d404ef (127.3.174.131:45361)
I20260808 05:00:03.601379 23411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:47415
I20260808 05:00:03.608235 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:00:03.627195 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.610730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.3.174.190:35155)
0808 05:00:03.610776 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:03.610778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:03.613932 (+ 3154us) server_negotiation.cc:407] Connection header received
0808 05:00:03.614060 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:03.614062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:03.614073 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:03.614101 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:03.615025 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.615806 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.622180 (+ 6374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.622490 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.622540 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.624199 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:03.624201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:03.624201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:03.624208 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:03.625867 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:03.626265 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:03.626266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:03.626266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:03.626324 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:03.626568 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:03.626569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:03.626570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:03.626717 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:03.626799 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:03.627009 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 05:00:03.627091 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 05:00:03.629251 23411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:00:03.631724 23411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:00:03.645692 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.635529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:36225 (local address 127.0.0.1:55762)
0808 05:00:03.637889 (+ 2360us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:03.637898 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:03.637942 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:03.638485 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:03.638489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:03.638491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:03.638561 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.638563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.642881 (+ 4318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.642882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.643787 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.643791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.645371 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.645372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.645455 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.645477 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 05:00:03.645546 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 05:00:03.645594 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2339,"mutex_wait_us":5,"thread_start_us":105,"threads_started":1}
I20260808 05:00:03.646029 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.635740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55762 (local address 127.3.174.130:36225)
0808 05:00:03.637468 (+ 1728us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:03.637474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:03.637994 (+ 520us) server_negotiation.cc:407] Connection header received
0808 05:00:03.638059 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:03.638061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:03.638078 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:03.638145 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:03.638640 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.641852 (+ 3212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.643910 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.644564 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.644631 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.644759 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:03.645902 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0808 05:00:03.645945 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1680,"thread_start_us":136,"threads_started":1}
I20260808 05:00:03.646791 23876 tablet_service.cc:1467] Processing CreateTablet for tablet 920223dcf16a429798303949acf4fa65 (DEFAULT_TABLE table=test-table [id=d8fd8cf575cf41d19149e7f5afd46315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:00:03.647145 23876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920223dcf16a429798303949acf4fa65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:03.650743 24251 tablet_bootstrap.cc:492] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b: Bootstrap starting.
I20260808 05:00:03.652352 24251 tablet_bootstrap.cc:654] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:03.652551 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.635260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:36361 (local address 127.0.0.1:48040)
0808 05:00:03.637632 (+ 2372us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:03.637676 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:03.637755 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:03.641960 (+ 4205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:03.641961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:03.641965 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:03.642042 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.642044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.645356 (+ 3312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.645358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.651271 (+ 5913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.651275 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.652328 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.652329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.652394 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.652400 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 05:00:03.652435 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 05:00:03.652480 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2333,"thread_start_us":96,"threads_started":1}
I20260808 05:00:03.652942 24251 log.cc:824] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:03.654646 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.635362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48040 (local address 127.3.174.129:36361)
0808 05:00:03.637546 (+ 2184us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:03.637550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:03.638251 (+ 701us) server_negotiation.cc:407] Connection header received
0808 05:00:03.638290 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:03.638292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:03.638308 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:03.638377 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:03.642132 (+ 3755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.645234 (+ 3102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.651373 (+ 6139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.652172 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.654417 (+ 2245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.654498 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:03.654524 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 05:00:03.654572 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2151,"thread_start_us":73,"threads_started":1}
I20260808 05:00:03.655506 23642 tablet_service.cc:1467] Processing CreateTablet for tablet 920223dcf16a429798303949acf4fa65 (DEFAULT_TABLE table=test-table [id=d8fd8cf575cf41d19149e7f5afd46315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:00:03.655900 23642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920223dcf16a429798303949acf4fa65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:03.657043 24242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.635725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:45361 (local address 127.0.0.1:39590)
0808 05:00:03.645686 (+ 9961us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:03.652802 (+ 7116us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:03.652870 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:03.653270 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:03.653271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:03.653274 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:03.653346 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.653348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.654302 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.654303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.655339 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.655342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.656802 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.656804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.656870 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.656875 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:00:03.656924 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 05:00:03.656967 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9937,"thread_start_us":81,"threads_started":1}
I20260808 05:00:03.657336 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.635998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39590 (local address 127.3.174.131:45361)
0808 05:00:03.642329 (+ 6331us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:03.642336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:03.652943 (+ 10607us) server_negotiation.cc:407] Connection header received
0808 05:00:03.652996 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:03.652999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:03.653071 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:03.653152 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:03.653428 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.654215 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.655801 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.656593 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.656663 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.656763 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:03.657142 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 05:00:03.657200 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6282,"thread_start_us":110,"threads_started":1}
I20260808 05:00:03.659435 24254 tablet_bootstrap.cc:492] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c: Bootstrap starting.
I20260808 05:00:03.660274 24107 tablet_service.cc:1467] Processing CreateTablet for tablet 920223dcf16a429798303949acf4fa65 (DEFAULT_TABLE table=test-table [id=d8fd8cf575cf41d19149e7f5afd46315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:00:03.660710 24107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920223dcf16a429798303949acf4fa65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:03.660784 24254 tablet_bootstrap.cc:654] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:03.661518 24254 log.cc:824] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:03.662777 24254 tablet_bootstrap.cc:492] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c: No bootstrap required, opened a new log
I20260808 05:00:03.662842 24254 ts_tablet_manager.cc:1397] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 05:00:03.666575 24254 raft_consensus.cc:348] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } }
I20260808 05:00:03.666848 24254 raft_consensus.cc:374] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:03.666951 24254 raft_consensus.cc:729] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 109c5c4d42bb46329bfc697a403eaa8c, State: Initialized, Role: FOLLOWER
I20260808 05:00:03.667153 24254 consensus_queue.cc:260] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } }
I20260808 05:00:03.667443 24251 tablet_bootstrap.cc:492] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b: No bootstrap required, opened a new log
I20260808 05:00:03.667531 24251 ts_tablet_manager.cc:1397] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260808 05:00:03.667663 24254 ts_tablet_manager.cc:1428] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 05:00:03.667856 23766 heartbeater.cc:503] Master 127.3.174.190:35155 was elected leader, sending a full tablet report...
I20260808 05:00:03.667972 24258 tablet_bootstrap.cc:492] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: Bootstrap starting.
I20260808 05:00:03.669414 24258 tablet_bootstrap.cc:654] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:03.670802 24258 log.cc:824] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:03.671546 24251 raft_consensus.cc:348] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } }
I20260808 05:00:03.671731 24258 tablet_bootstrap.cc:492] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: No bootstrap required, opened a new log
I20260808 05:00:03.671737 24251 raft_consensus.cc:374] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:03.671777 24251 raft_consensus.cc:729] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 644de1260b5944fd87078852b5cbae6b, State: Initialized, Role: FOLLOWER
I20260808 05:00:03.671806 24258 ts_tablet_manager.cc:1397] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 05:00:03.672050 24251 consensus_queue.cc:260] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } }
I20260808 05:00:03.672408 24251 ts_tablet_manager.cc:1428] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260808 05:00:03.672552 23989 heartbeater.cc:503] Master 127.3.174.190:35155 was elected leader, sending a full tablet report...
I20260808 05:00:03.675064 24258 raft_consensus.cc:348] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } }
I20260808 05:00:03.675241 24258 raft_consensus.cc:374] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:03.675289 24258 raft_consensus.cc:729] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47357dc2a79d456fbe596826e4d404ef, State: Initialized, Role: FOLLOWER
I20260808 05:00:03.677717 24258 consensus_queue.cc:260] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } }
I20260808 05:00:03.678071 24258 ts_tablet_manager.cc:1428] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260808 05:00:03.678218 24218 heartbeater.cc:503] Master 127.3.174.190:35155 was elected leader, sending a full tablet report...
W20260808 05:00:03.714285 23768 tablet.cc:2369] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:00:03.730432 24261 raft_consensus.cc:482] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:00:03.730614 24261 raft_consensus.cc:504] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } }
I20260808 05:00:03.731230 24261 leader_election.cc:288] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 109c5c4d42bb46329bfc697a403eaa8c (127.3.174.129:36361), 47357dc2a79d456fbe596826e4d404ef (127.3.174.131:45361)
I20260808 05:00:03.739933 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.731592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:36361 (local address 127.3.174.130:32851)
0808 05:00:03.732052 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:03.732065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:03.732116 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:03.732673 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:03.732675 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:03.732677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:03.732866 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.732872 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.733880 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.733881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.738565 (+ 4684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.738569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.739606 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.739607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.739743 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.739748 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:00:03.739804 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 05:00:03.739852 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436}
I20260808 05:00:03.740694 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.732342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:32851 (local address 127.3.174.129:36361)
0808 05:00:03.732469 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:03.732471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:03.732481 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:00:03.732510 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:03.732511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:03.732523 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:03.732575 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:03.732964 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.733759 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.738671 (+ 4912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.739483 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.739559 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.739570 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:03.740579 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0808 05:00:03.740627 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 05:00:03.741097 23662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920223dcf16a429798303949acf4fa65" candidate_uuid: "644de1260b5944fd87078852b5cbae6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "109c5c4d42bb46329bfc697a403eaa8c" is_pre_election: true
I20260808 05:00:03.741389 23662 raft_consensus.cc:2454] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 644de1260b5944fd87078852b5cbae6b in term 0.
I20260808 05:00:03.741823 23807 leader_election.cc:302] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 109c5c4d42bb46329bfc697a403eaa8c, 644de1260b5944fd87078852b5cbae6b; no voters:
I20260808 05:00:03.742046 24261 raft_consensus.cc:2781] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:00:03.742120 24261 raft_consensus.cc:482] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:00:03.742146 24261 raft_consensus.cc:3037] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:03.743320 24261 raft_consensus.cc:504] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } }
I20260808 05:00:03.743466 24261 leader_election.cc:288] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [CANDIDATE]: Term 1 election: Requested vote from peers 109c5c4d42bb46329bfc697a403eaa8c (127.3.174.129:36361), 47357dc2a79d456fbe596826e4d404ef (127.3.174.131:45361)
I20260808 05:00:03.743780 23662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920223dcf16a429798303949acf4fa65" candidate_uuid: "644de1260b5944fd87078852b5cbae6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "109c5c4d42bb46329bfc697a403eaa8c"
I20260808 05:00:03.743860 23662 raft_consensus.cc:3037] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:03.745100 23662 raft_consensus.cc:2454] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 644de1260b5944fd87078852b5cbae6b in term 1.
I20260808 05:00:03.745397 23807 leader_election.cc:302] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 109c5c4d42bb46329bfc697a403eaa8c, 644de1260b5944fd87078852b5cbae6b; no voters:
I20260808 05:00:03.746572 24261 raft_consensus.cc:2781] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:00:03.746766 24261 raft_consensus.cc:686] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 1 LEADER]: Becoming Leader. State: Replica: 644de1260b5944fd87078852b5cbae6b, State: Running, Role: LEADER
I20260808 05:00:03.746868 24261 consensus_queue.cc:237] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } }
I20260808 05:00:03.750289 23411 catalog_manager.cc:5495] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 644de1260b5944fd87078852b5cbae6b (127.3.174.130). New cstate: current_term: 1 leader_uuid: "644de1260b5944fd87078852b5cbae6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "644de1260b5944fd87078852b5cbae6b" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 } health_report { overall_health: UNKNOWN } } }
I20260808 05:00:03.751672 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.731721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:45361 (local address 127.3.174.130:41039)
0808 05:00:03.740009 (+ 8288us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:03.740022 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:03.740067 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:03.745526 (+ 5459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:03.745528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:03.745530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:03.745570 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.745572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.747555 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.747557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.748629 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:03.748632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.751443 (+ 2811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.751444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:03.751510 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.751513 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:00:03.751566 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 05:00:03.751604 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8277,"thread_start_us":85,"threads_started":1}
I20260808 05:00:03.751992 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.732407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:41039 (local address 127.3.174.131:45361)
0808 05:00:03.732635 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:03.732637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:03.740121 (+ 7484us) server_negotiation.cc:407] Connection header received
0808 05:00:03.740151 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:03.740152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:03.740162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:03.740217 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:03.745664 (+ 5447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.746417 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.748736 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.749754 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.749817 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.749829 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:03.751871 (+ 2042us) server_negotiation.cc:299] Negotiation successful
0808 05:00:03.751921 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260808 05:00:03.752626 24140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920223dcf16a429798303949acf4fa65" candidate_uuid: "644de1260b5944fd87078852b5cbae6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47357dc2a79d456fbe596826e4d404ef"
I20260808 05:00:03.752794 24140 raft_consensus.cc:3037] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:03.753988 24140 raft_consensus.cc:2454] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 644de1260b5944fd87078852b5cbae6b in term 1.
I20260808 05:00:03.752632 24115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920223dcf16a429798303949acf4fa65" candidate_uuid: "644de1260b5944fd87078852b5cbae6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47357dc2a79d456fbe596826e4d404ef" is_pre_election: true
I20260808 05:00:03.754428 24115 raft_consensus.cc:2362] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 644de1260b5944fd87078852b5cbae6b in term 1. Re-sending same reply.
W20260808 05:00:03.773510 23990 tablet.cc:2369] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:00:03.784835 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.781074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55764 (local address 127.3.174.130:36225)
0808 05:00:03.781123 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:03.781125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:03.781132 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:00:03.781191 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:03.781191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:03.781218 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:03.781305 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:00:03.781571 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.782334 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.783483 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:03.783874 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:03.783953 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:03.784192 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:00:03.784335 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:00:03.784415 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:03.784666 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 05:00:03.784691 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 05:00:03.787297 24115 raft_consensus.cc:1261] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [term 1 FOLLOWER]: Refusing update from remote peer 644de1260b5944fd87078852b5cbae6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:00:03.787324 23662 raft_consensus.cc:1261] T 920223dcf16a429798303949acf4fa65 P 109c5c4d42bb46329bfc697a403eaa8c [term 1 FOLLOWER]: Refusing update from remote peer 644de1260b5944fd87078852b5cbae6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:00:03.787719 24264 consensus_queue.cc:1035] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "109c5c4d42bb46329bfc697a403eaa8c" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:00:03.787997 24264 consensus_queue.cc:1035] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "47357dc2a79d456fbe596826e4d404ef" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 45361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:00:03.794710 24267 mvcc.cc:205] Tried to move back new op lower bound from 7316132674709831680 to 7316132674547847168. Current Snapshot: MvccSnapshot[applied={T|T < 7316132674709831680}]
I20260808 05:00:03.795590 24269 mvcc.cc:205] Tried to move back new op lower bound from 7316132674709831680 to 7316132674547847168. Current Snapshot: MvccSnapshot[applied={T|T < 7316132674709831680}]
Aug 08 05:00:03 dist-test-slave-gjw7 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:03.798650 24268 mvcc.cc:205] Tried to move back new op lower bound from 7316132674709831680 to 7316132674547847168. Current Snapshot: MvccSnapshot[applied={T|T < 7316132674709831680 or (T in {7316132674709831680})}]
I20260808 05:00:03.801524 23411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:00:03.801621 23411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:00:03.802448 23411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ec88846ecf04188a6f66f3790c586b6: Sending DeleteTablet for 3 replicas of tablet 920223dcf16a429798303949acf4fa65
I20260808 05:00:03.803010 23642 tablet_service.cc:1514] Processing DeleteTablet for tablet 920223dcf16a429798303949acf4fa65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:00:03 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:48040
I20260808 05:00:03.803287 23876 tablet_service.cc:1514] Processing DeleteTablet for tablet 920223dcf16a429798303949acf4fa65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:00:03 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:55762
I20260808 05:00:03.803606 24107 tablet_service.cc:1514] Processing DeleteTablet for tablet 920223dcf16a429798303949acf4fa65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:00:03 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:39590
I20260808 05:00:03.804323 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 23540
I20260808 05:00:03.830936 24283 tablet_replica.cc:330] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: stopping tablet replica
I20260808 05:00:03.831224 24282 tablet_replica.cc:330] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b: stopping tablet replica
I20260808 05:00:03.832346 24283 raft_consensus.cc:2229] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:00:03.832579 24282 raft_consensus.cc:2229] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:00:03.832772 24283 raft_consensus.cc:2258] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:00:03.832980 24282 raft_consensus.cc:2258] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:00:03.833786 24283 ts_tablet_manager.cc:1905] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:00:03.833786 24282 ts_tablet_manager.cc:1905] T 920223dcf16a429798303949acf4fa65 P 644de1260b5944fd87078852b5cbae6b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 05:00:03.841531 23390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:36361 (error 108)
I20260808 05:00:03.843775 24283 ts_tablet_manager.cc:1918] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:00:03.843890 24283 log.cc:1196] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/920223dcf16a429798303949acf4fa65
I20260808 05:00:03.844133 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 23777
I20260808 05:00:03.854643 24283 ts_tablet_manager.cc:1939] T 920223dcf16a429798303949acf4fa65 P 47357dc2a79d456fbe596826e4d404ef: Deleting consensus metadata
I20260808 05:00:03.855268 23391 catalog_manager.cc:4841] TS 47357dc2a79d456fbe596826e4d404ef (127.3.174.131:45361): tablet 920223dcf16a429798303949acf4fa65 (table test-table [id=d8fd8cf575cf41d19149e7f5afd46315]) successfully deleted
I20260808 05:00:03.860497 24242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.860340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:36361 (local address 127.0.0.1:48048)
0808 05:00:03.860401 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:03.860452 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:36361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 05:00:03.860571 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:03.860495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:36225 (local address 127.0.0.1:55770)
0808 05:00:03.860531 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:03.860552 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.130:36225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 05:00:03.860653 23390 catalog_manager.cc:4584] TS 109c5c4d42bb46329bfc697a403eaa8c (127.3.174.129:36361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 920223dcf16a429798303949acf4fa65: Network error: Client connection negotiation failed: client connection to 127.3.174.129:36361: connect: Connection refused (error 111)
I20260808 05:00:03.860992 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 23996
I20260808 05:00:03.872427 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 23367
I20260808 05:00:03.901115 3770 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 05:00:04.255702 3770 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 05:00:04.255820 3770 mini_ranger.cc:66] Stopping Ranger...
I20260808 05:00:04.602041 3770 mini_ranger.cc:68] Stopped Ranger
2026-08-08 05:00:04.602 UTC [22704] LOG: received smart shutdown request
2026-08-08 05:00:04.606 UTC [22704] LOG: background worker "logical replication launcher" (PID 22717) exited with exit code 1
2026-08-08 05:00:04.606 UTC [22709] LOG: shutting down
2026-08-08 05:00:04.763 UTC [22704] LOG: database system is shut down
2026-08-08T05:00:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49376 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:00:04 dist-test-slave-gjw7 krb5kdc[24296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:00:04 dist-test-slave-gjw7 krb5kdc[24296](info): set up 2 sockets
Aug 08 05:00:04 dist-test-slave-gjw7 krb5kdc[24296](info): commencing operation
krb5kdc: starting...
W20260808 05:00:06.078840 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:00:06 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:00:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:00:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.174.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.174.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/rangeradmin_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/rangeradmin_127.3.174.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.174.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.174.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/rangerlookup_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/rangerlookup_127.3.174.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.174.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.174.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/HTTP_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/HTTP_127.3.174.148@KRBTEST.COM.keytab.
I20260808 05:00:06.262120 3770 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskihVxiI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/postgres -l logfile start
2026-08-08 05:00:08.564 UTC [24350] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 05:00:08.564 UTC [24350] LOG: listening on IPv4 address "127.3.174.148", port 54153
2026-08-08 05:00:08.568 UTC [24350] LOG: listening on Unix socket "/tmp/.s.PGSQL.54153"
2026-08-08 05:00:08.588 UTC [24356] LOG: database system was shut down at 2026-08-08 05:00:07 UTC
2026-08-08 05:00:08.592 UTC [24350] LOG: database system is ready to accept connections
I20260808 05:00:09.689568 3770 mini_postgres.cc:96] Postgres bound to 54153
2026-08-08 05:00:09.696 UTC [24381] FATAL: database "slave" does not exist
127.3.174.148:54153 - accepting connections
I20260808 05:00:09.696861 3770 mini_ranger.cc:161] Starting Ranger...
I20260808 05:00:09.711675 3770 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 05:00:10.136585 3770 mini_ranger.cc:87] Created ranger Postgres database
I20260808 05:00:10.136672 3770 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-admin
2026-08-08 05:00:10,423 [I] DB FLAVOR :POSTGRES
2026-08-08 05:00:10,424 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 05:00:10,424 [I] Checking connection..
2026-08-08 05:00:10,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 05:00:10,690 [I] Checking connection passed.
2026-08-08 05:00:10,690 [I] --------- Verifying version history table ---------
2026-08-08 05:00:10,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 05:00:10,955 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 05:00:10,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 05:00:11,221 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 05:00:11,221 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 05:00:11,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 05:00:11.476 UTC [24474] WARNING: there is no transaction in progress
2026-08-08 05:00:11,485 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 05:00:11,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 05:00:11,747 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 05:00:11,748 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 05:00:11,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 05:00:12,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 05:00:12,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjw7.c.gcp-upstream.internal','N') ;"
2026-08-08 05:00:12,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 05:00:12,730 [I] Table x_portal_user does not exist in database ranger
2026-08-08 05:00:12,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 05:00:12,982 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 05:00:12,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 05:00:13,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 05:00:13,505 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 05:00:13,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 05:00:14.428 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.442 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.458 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.473 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.489 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.506 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.559 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.565 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.578 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.589 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.598 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.610 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.620 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.626 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.632 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:14.639 UTC [24647] WARNING: there is no transaction in progress
2026-08-08 05:00:15,220 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 05:00:15,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 05:00:15,466 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 05:00:15,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 05:00:15,712 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 05:00:15,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 05:00:15,952 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 05:00:15,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 05:00:16,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 05:00:16,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjw7.c.gcp-upstream.internal';"
2026-08-08 05:00:16,692 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 05:00:16,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 05:00:16,941 [I] DB_PATCHES have already been applied
I20260808 05:00:16.946919 3770 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-admin:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskihVxiI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 05:00:16.946985 3770 mini_ranger.cc:193] Using host: 127.3.174.148
I20260808 05:00:16.950995 3770 mini_ranger.cc:227] Ranger admin URL: http://127.3.174.148:38293
Aug 08, 2026 5:00:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:00:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:00:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 5:00:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 5:00:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 5:00:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 5:00:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 05:00:18.255343 3770 mini_ranger.cc:160] Time spent starting Ranger: real 8.558s user 0.000s sys 0.010s
Aug 08, 2026 5:00:38 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 5:00:38 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 5:00:38 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 5:00:38 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 5:00:39 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 05:00:40.241981 3770 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.174.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.174.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/rangerkms_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.174.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/rangerkms_127.3.174.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.174.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.174.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.174.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/HTTP_127.3.174.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.174.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/HTTP_127.3.174.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 05:00:40 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165240, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 05:00:40.324656 3770 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 05:00:40.333884 3770 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 05:00:41.166697 3770 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 05:00:41.166813 3770 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-kms
I20260808 05:00:41.166831 3770 mini_ranger_kms.cc:227] Using postgres at 127.3.174.148:54153
2026-08-08 05:00:41,417 [I] DB FLAVOR :POSTGRES
2026-08-08 05:00:41,417 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 05:00:41,417 [I] Checking connection
2026-08-08 05:00:41,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 05:00:41,663 [I] connection success
2026-08-08 05:00:41,664 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 05:00:41,664 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 05:00:41,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 05:00:41,897 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 05:00:41,897 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 05:00:41,898 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 05:00:41,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.174.148:54153/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 05:00:42,174 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 05:00:42.683115 3770 mini_ranger_kms.cc:310] Created kms service
I20260808 05:00:42.881027 3770 mini_ranger_kms.cc:326] Created kudu user
I20260808 05:00:42.924489 3770 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 05:00:42 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165240, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:43.310312 3770 mini_ranger_kms.cc:384] Added ranger policy
I20260808 05:00:43.310510 3770 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-kms:/tmp/dist-test-taskihVxiI/build/release/bin/postgresql.jar:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskihVxiI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 05:00:43.310530 3770 mini_ranger_kms.cc:240] Using host: 127.3.174.148
I20260808 05:00:43.314675 3770 mini_ranger_kms.cc:276] Ranger KMS PID: 24968
I20260808 05:00:43.314752 3770 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.174.148:50227
Aug 08, 2026 5:00:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:00:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:00:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 5:00:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 5:00:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 5:00:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 05:00:45 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786165245, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.174.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:45.950706 3770 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.626s user 0.015s sys 0.218s
I20260808 05:00:45.950800 3770 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 05:00:45.950848 3770 mini_ranger_kms.cc:401] 127.3.174.148:50227/kms/v1/keys
Aug 08 05:00:46 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165245, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.174.148@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
Aug 08 05:00:46 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165245, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.174.148@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
Aug 08 05:00:48 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:00:48.802130 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45379
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:45379
--ranger_config_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:50227/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:48.887632 25060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:48.887823 25060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:48.887852 25060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:48.888963 25060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:00:48.889004 25060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:48.889024 25060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:48.889039 25060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:00:48.889055 25060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:00:48.890599 25060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.174.148:50227/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:45379
--ranger_config_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:48.890815 25060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:48.891072 25060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 05:00:48.895998 25060 server_base.cc:1034] running on GCE node
W20260808 05:00:48.895990 25083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:48.895956 25069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:48.896034 25070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:48.896399 25060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:48.896641 25060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:48.897766 25060 hybrid_clock.cc:648] HybridClock initialized: now 1786165248897754 us; error 32 us; skew 500 ppm
Aug 08 05:00:48 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165248, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:48.900147 25060 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:00:48.900525 25060 webserver.cc:460] Webserver started at http://127.3.174.190:42435/ using document root <none> and password file <none>
I20260808 05:00:48.900802 25060 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:48.900851 25060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:48.900978 25060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 05:00:48 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165248, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:48.966140 25060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "041f2c1fa7604f71b90077dde309c122"
format_stamp: "Formatted at 2026-08-08 05:00:48 on dist-test-slave-gjw7"
server_key: "6b6060085262ae497efcd903345ce36c"
server_key_iv: "2cdd0b728a959ce2f8c1d8103beeb5cc"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:48.966617 25060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "041f2c1fa7604f71b90077dde309c122"
format_stamp: "Formatted at 2026-08-08 05:00:48 on dist-test-slave-gjw7"
server_key: "6b6060085262ae497efcd903345ce36c"
server_key_iv: "2cdd0b728a959ce2f8c1d8103beeb5cc"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:48.968258 25060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:00:48.983343 25093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:48.983525 25060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:00:48.983613 25060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "041f2c1fa7604f71b90077dde309c122"
format_stamp: "Formatted at 2026-08-08 05:00:48 on dist-test-slave-gjw7"
server_key: "6b6060085262ae497efcd903345ce36c"
server_key_iv: "2cdd0b728a959ce2f8c1d8103beeb5cc"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:48.983714 25060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:49.005208 25060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:49.007364 25060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:49.007539 25060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:49.012765 25060 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:45379
I20260808 05:00:49.012796 25184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:45379 every 8 connection(s)
I20260808 05:00:49.013242 25060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:00:49.014022 25185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:49.016317 25185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Bootstrap starting.
I20260808 05:00:49.016901 25185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:49.017161 25185 log.cc:824] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:49.017978 25185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: No bootstrap required, opened a new log
I20260808 05:00:49.019637 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 25060
I20260808 05:00:49.019758 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:00:49.019955 3770 external_mini_cluster.cc:1428] Setting key 414a4a227848846354d6f3291e76c946
I20260808 05:00:49.020177 25185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } }
I20260808 05:00:49.020378 25185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:49.020440 25185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 041f2c1fa7604f71b90077dde309c122, State: Initialized, Role: FOLLOWER
I20260808 05:00:49.020576 25185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } }
I20260808 05:00:49.020763 25185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:00:49.020815 25185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:00:49.020855 25185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:49.021535 25185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } }
I20260808 05:00:49.021641 25185 leader_election.cc:302] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 041f2c1fa7604f71b90077dde309c122; no voters:
I20260808 05:00:49.021797 25185 leader_election.cc:288] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:00:49.022145 25185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:00:49.021837 25200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:00:49.023522 25200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 1 LEADER]: Becoming Leader. State: Replica: 041f2c1fa7604f71b90077dde309c122, State: Running, Role: LEADER
I20260808 05:00:49.023653 25200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } }
I20260808 05:00:49.024288 25203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "041f2c1fa7604f71b90077dde309c122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } } }
I20260808 05:00:49.024387 25203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:49.024348 25223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 041f2c1fa7604f71b90077dde309c122. Latest consensus state: current_term: 1 leader_uuid: "041f2c1fa7604f71b90077dde309c122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } } }
I20260808 05:00:49.024427 25223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:49.024304 25185 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 05:00:49.027872 25228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:00:49.028630 25228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 05:00:49 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:49.031760 25228 catalog_manager.cc:1347] Generated new cluster ID: 2ebfed64b10c4713b7328874ef12bd02
I20260808 05:00:49.031808 25228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:00:49.034495 25193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:49.020469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.3.174.190:45379)
0808 05:00:49.020649 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:49.020658 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:49.020725 (+ 67us) server_negotiation.cc:407] Connection header received
0808 05:00:49.020890 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:49.020898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:49.020959 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:49.021114 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:49.021936 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:49.023405 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:49.024459 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:49.026887 (+ 2428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:49.027081 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:49.029768 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:49.029779 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:49.029779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:49.029797 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:49.032246 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:49.032664 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:49.032668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:49.032668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:49.032792 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:49.033015 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:49.033016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:49.033016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:49.033407 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:49.033551 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:49.034007 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 05:00:49.034216 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260808 05:00:49.053649 25228 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:00:49.054498 25228 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:00:49.068563 25228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Generated new TSK 0
I20260808 05:00:49.068948 25228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 05:00:49.773761 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:50227/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45379
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:49.903007 25274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:49.903343 25274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:49.903388 25274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:49.905267 25274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:49.905351 25274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:49.905393 25274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:00:49.907739 25274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.174.148:50227/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:49.908036 25274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:49.908349 25274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:49.914099 25284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:49.914177 25286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:49.914119 25283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:49.914402 25274 server_base.cc:1034] running on GCE node
I20260808 05:00:49.915123 25274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:49.915457 25274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:49.916658 25274 hybrid_clock.cc:648] HybridClock initialized: now 1786165249916646 us; error 55 us; skew 500 ppm
Aug 08 05:00:49 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165249, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:49.920013 25274 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:00:49.920579 25274 webserver.cc:460] Webserver started at http://127.3.174.129:33957/ using document root <none> and password file <none>
I20260808 05:00:49.920884 25274 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:49.920955 25274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:49.921103 25274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 05:00:49 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165249, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:49.942099 25274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "a47bf364fb4b42369fc72335207ebf55"
format_stamp: "Formatted at 2026-08-08 05:00:49 on dist-test-slave-gjw7"
server_key: "a727b9279da06424bf50c15c61c6342e"
server_key_iv: "524c57f7d73cdbe840ff56a4f65eae81"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:49.942685 25274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "a47bf364fb4b42369fc72335207ebf55"
format_stamp: "Formatted at 2026-08-08 05:00:49 on dist-test-slave-gjw7"
server_key: "a727b9279da06424bf50c15c61c6342e"
server_key_iv: "524c57f7d73cdbe840ff56a4f65eae81"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:49.945363 25274 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 05:00:49.959736 25295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:49.959920 25274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:00:49.960026 25274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "a47bf364fb4b42369fc72335207ebf55"
format_stamp: "Formatted at 2026-08-08 05:00:49 on dist-test-slave-gjw7"
server_key: "a727b9279da06424bf50c15c61c6342e"
server_key_iv: "524c57f7d73cdbe840ff56a4f65eae81"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:49.960137 25274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:49.973327 25274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:49.976341 25274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:49.976517 25274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:49.976780 25274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:49.977167 25274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:49.977227 25274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:49.977260 25274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:49.977284 25274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:49.988402 25274 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:33265
I20260808 05:00:49.988488 25515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:33265 every 8 connection(s)
I20260808 05:00:49.988981 25274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:00:49.992815 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 25274
I20260808 05:00:49.992934 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:00:49.993129 3770 external_mini_cluster.cc:1428] Setting key 8d0d930db78a4e0e957aeb764bec1e04
Aug 08 05:00:49 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165249, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:50.002274 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:49.989861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45379 (local address 127.3.174.129:58039)
0808 05:00:49.990082 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:49.990128 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:49.990797 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:49.991081 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:49.991085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:49.991424 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:49.991745 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:49.991753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:49.992830 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:49.992831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:49.994215 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:49.994219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:49.995772 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:49.995774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:49.995950 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:49.996616 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:49.996628 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:49.998381 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:50.000459 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:50.000467 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:50.000470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:50.000763 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:50.000968 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:50.000968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:50.000969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:50.001098 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:50.001440 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:50.001448 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:50.001633 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 05:00:50.001748 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 05:00:50.002061 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":111,"threads_started":1}
I20260808 05:00:50.002636 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:49.989882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:58039 (local address 127.3.174.190:45379)
0808 05:00:49.990036 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:49.990039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:49.990301 (+ 262us) server_negotiation.cc:407] Connection header received
0808 05:00:49.990898 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:49.990900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:49.990915 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:49.990993 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:49.991873 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:49.992716 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:49.994316 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:49.995596 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:49.995665 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:49.998489 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:49.998491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:49.998491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:49.998500 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:50.000367 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:50.000844 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:50.000846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:50.000846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:50.000908 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:50.001164 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:50.001166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:50.001166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:50.001350 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:50.001412 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:50.002441 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0808 05:00:50.002538 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260808 05:00:50.003050 25517 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45379
I20260808 05:00:50.003166 25517 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:50.003387 25517 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
I20260808 05:00:50.003978 25119 ts_manager.cc:194] Registered new tserver with Master: a47bf364fb4b42369fc72335207ebf55 (127.3.174.129:33265)
I20260808 05:00:50.004623 25119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:58039
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 05:00:50.033870 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:50227/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45379
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:50.165469 25528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:50.165732 25528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:50.165786 25528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:50.167546 25528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:50.167614 25528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:50.167656 25528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:00:50.169914 25528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.174.148:50227/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:50.170192 25528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:50.170514 25528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 05:00:50.176400 25544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:50.176483 25546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:50.176771 25528 server_base.cc:1034] running on GCE node
W20260808 05:00:50.176416 25551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:50.177047 25528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:50.177356 25528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:50.178542 25528 hybrid_clock.cc:648] HybridClock initialized: now 1786165250178521 us; error 31 us; skew 500 ppm
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:50.181984 25528 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:00:50.182478 25528 webserver.cc:460] Webserver started at http://127.3.174.130:44101/ using document root <none> and password file <none>
I20260808 05:00:50.182704 25528 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:50.182760 25528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:50.182894 25528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:50.196085 25528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c"
format_stamp: "Formatted at 2026-08-08 05:00:50 on dist-test-slave-gjw7"
server_key: "a96a54e04946c6d637446c14c294a8eb"
server_key_iv: "2e1efa05f79e611b72a11f71485233f5"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:50.196588 25528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c"
format_stamp: "Formatted at 2026-08-08 05:00:50 on dist-test-slave-gjw7"
server_key: "a96a54e04946c6d637446c14c294a8eb"
server_key_iv: "2e1efa05f79e611b72a11f71485233f5"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:50.198661 25528 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 05:00:50.221562 25565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.221874 25528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:00:50.221989 25528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c"
format_stamp: "Formatted at 2026-08-08 05:00:50 on dist-test-slave-gjw7"
server_key: "a96a54e04946c6d637446c14c294a8eb"
server_key_iv: "2e1efa05f79e611b72a11f71485233f5"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:50.222092 25528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:50.239818 25528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:50.242434 25528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:50.242636 25528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:50.242962 25528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:50.243432 25528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:50.243485 25528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.243558 25528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:50.243602 25528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.252700 25528 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:44441
I20260808 05:00:50.252789 25759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:44441 every 8 connection(s)
I20260808 05:00:50.253216 25528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 05:00:50.260437 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 25528
I20260808 05:00:50.260553 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:00:50.260756 3770 external_mini_cluster.cc:1428] Setting key 83407eca636cecfc1d6e463ee8be82c1
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:50.269807 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:50.254912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45379 (local address 127.3.174.130:54589)
0808 05:00:50.255378 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:50.255429 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:50.256146 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:50.256577 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:50.256582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:50.256877 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:50.257186 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:50.257193 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.258318 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.258319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:50.259651 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:50.259655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.260273 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.260274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:50.260424 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:50.264072 (+ 3648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:50.264085 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:50.266036 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:50.267941 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:50.267950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:50.267954 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:50.268249 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:50.268508 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:50.268509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:50.268510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:50.268676 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:50.269016 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:50.269025 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:50.269220 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 05:00:50.269356 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 05:00:50.269537 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":99,"threads_started":1}
I20260808 05:00:50.270151 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:50.255250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:54589 (local address 127.3.174.190:45379)
0808 05:00:50.255299 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:50.255301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:50.255524 (+ 223us) server_negotiation.cc:407] Connection header received
0808 05:00:50.256300 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:50.256302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:50.256320 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:50.256350 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:50.257350 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.258182 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.259757 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.260083 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.260133 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:50.266176 (+ 6043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:50.266178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:50.266178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:50.266187 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:50.267834 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:50.268336 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:50.268337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:50.268338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:50.268425 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:50.268757 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:50.268758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:50.268758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:50.268926 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:50.268987 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:50.269969 (+ 982us) server_negotiation.cc:299] Negotiation successful
0808 05:00:50.270065 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 05:00:50.270480 25761 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45379
I20260808 05:00:50.270596 25761 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:50.270833 25761 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
I20260808 05:00:50.271231 25119 ts_manager.cc:194] Registered new tserver with Master: 8da51ccd71764ec19ebfc1c5c39fcf8c (127.3.174.130:44441)
I20260808 05:00:50.271724 25119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:54589
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 05:00:50.300103 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:50227/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45379
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:50.435024 25767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:50.435326 25767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:50.435370 25767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:50.437237 25767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:50.437333 25767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:50.437389 25767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:00:50.439833 25767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.174.148:50227/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:50.440131 25767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:50.440413 25767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:50.446738 25782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:50.446770 25781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:50.446754 25786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:50.447269 25767 server_base.cc:1034] running on GCE node
I20260808 05:00:50.447479 25767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:50.447741 25767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:50.448920 25767 hybrid_clock.cc:648] HybridClock initialized: now 1786165250448898 us; error 32 us; skew 500 ppm
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:50.452368 25767 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:00:50.452979 25767 webserver.cc:460] Webserver started at http://127.3.174.131:43047/ using document root <none> and password file <none>
I20260808 05:00:50.453277 25767 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:50.453377 25767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:50.453519 25767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:50.468218 25767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "5aa719ea0c414a599711ebeca2f98ff6"
format_stamp: "Formatted at 2026-08-08 05:00:50 on dist-test-slave-gjw7"
server_key: "83bb35fabbaf7df5f7f77bb23b21dde7"
server_key_iv: "779ef91a5187d5b749d2016635c888bc"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:50.468760 25767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "5aa719ea0c414a599711ebeca2f98ff6"
format_stamp: "Formatted at 2026-08-08 05:00:50 on dist-test-slave-gjw7"
server_key: "83bb35fabbaf7df5f7f77bb23b21dde7"
server_key_iv: "779ef91a5187d5b749d2016635c888bc"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:50.470918 25767 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 05:00:50.498739 25795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.498942 25767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 05:00:50.499046 25767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "5aa719ea0c414a599711ebeca2f98ff6"
format_stamp: "Formatted at 2026-08-08 05:00:50 on dist-test-slave-gjw7"
server_key: "83bb35fabbaf7df5f7f77bb23b21dde7"
server_key_iv: "779ef91a5187d5b749d2016635c888bc"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:50.499161 25767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:50.524662 25767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:50.526872 25767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:50.527024 25767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:50.527297 25767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:50.527683 25767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:50.527722 25767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.527745 25767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:50.527774 25767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.535029 25767 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:33965
I20260808 05:00:50.535353 26001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:33965 every 8 connection(s)
I20260808 05:00:50.535506 25767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:00:50.541620 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 25767
I20260808 05:00:50.541728 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 05:00:50.541917 3770 external_mini_cluster.cc:1428] Setting key a9911fd0918557dfdddd5198110bf7cd
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:50.545542 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:50.536646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:42345 (local address 127.3.174.190:45379)
0808 05:00:50.536698 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:50.536700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:50.536926 (+ 226us) server_negotiation.cc:407] Connection header received
0808 05:00:50.537568 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:50.537570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:50.537584 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:50.537638 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:50.538246 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.538753 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.539746 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.539969 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.540036 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:50.542710 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:50.542711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:50.542711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:50.542719 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:50.544191 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:50.544576 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:50.544578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:50.544578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:50.544642 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:50.544875 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:50.544876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:50.544876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:50.544996 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:50.545052 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:50.545359 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 05:00:50.545449 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 05:00:50.545723 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:50.536315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45379 (local address 127.3.174.131:42345)
0808 05:00:50.536787 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:50.536835 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:50.537456 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:50.537718 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:50.537723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:50.537937 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:50.538145 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:50.538151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.538843 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.538845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:50.539655 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:50.539657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.540128 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.540130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:50.540283 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:50.540832 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:50.540851 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:50.542615 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:50.544266 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:50.544273 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:50.544276 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:50.544508 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:50.544697 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:50.544698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:50.544699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:50.544812 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:50.545054 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:50.545061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:50.545202 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 05:00:50.545319 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 05:00:50.545485 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":95,"threads_started":1}
I20260808 05:00:50.546231 26002 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45379
I20260808 05:00:50.546337 26002 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:50.546536 26002 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
I20260808 05:00:50.546919 25119 ts_manager.cc:194] Registered new tserver with Master: 5aa719ea0c414a599711ebeca2f98ff6 (127.3.174.131:33965)
I20260808 05:00:50.547288 25119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:42345
I20260808 05:00:50.553880 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:00:50.554579 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 25274
I20260808 05:00:50.582474 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 25528
I20260808 05:00:50.592187 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 25767
I20260808 05:00:50.600701 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 25060
I20260808 05:00:50.611657 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45379
--webserver_interface=127.3.174.190
--webserver_port=42435
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:45379
--ranger_config_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:50227/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.174.148:50227/kms with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:50.694593 26006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:50.694790 26006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:50.694810 26006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:50.695888 26006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:00:50.695935 26006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:50.695972 26006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:50.696002 26006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:00:50.696017 26006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:00:50.697660 26006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.174.148:50227/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:45379
--ranger_config_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:50.697846 26006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:50.698011 26006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:50.702776 26014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:50.702802 26017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:50.702844 26006 server_base.cc:1034] running on GCE node
W20260808 05:00:50.702769 26013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:50.703228 26006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:50.703497 26006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:50.704632 26006 hybrid_clock.cc:648] HybridClock initialized: now 1786165250704609 us; error 39 us; skew 500 ppm
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:50.706974 26006 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:00:50.707430 26006 webserver.cc:460] Webserver started at http://127.3.174.190:42435/ using document root <none> and password file <none>
I20260808 05:00:50.707723 26006 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:50.707794 26006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:50.747454 26006 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:00:50.749564 26031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.749760 26006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 05:00:50.749871 26006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "041f2c1fa7604f71b90077dde309c122"
format_stamp: "Formatted at 2026-08-08 05:00:48 on dist-test-slave-gjw7"
server_key: "6b6060085262ae497efcd903345ce36c"
server_key_iv: "2cdd0b728a959ce2f8c1d8103beeb5cc"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:50.750272 26006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:50.763459 26006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:50.765570 26006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:50.765726 26006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:50.770179 26006 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:45379
I20260808 05:00:50.770228 26115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:45379 every 8 connection(s)
I20260808 05:00:50.770617 26006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:00:50.771893 26116 sys_catalog.cc:263] Verifying existing consensus state
I20260808 05:00:50.772542 26116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Bootstrap starting.
I20260808 05:00:50.777438 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 26006
I20260808 05:00:50.777766 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:50227/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:33265
--local_ip_for_outbound_sockets=127.3.174.129
--tserver_master_addrs=127.3.174.190:45379
--webserver_port=33957
--webserver_interface=127.3.174.129
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.174.148:50227/kms with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:00:50.780339 26116 log.cc:824] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:50.782200 26116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 05:00:50.782438 26116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Bootstrap complete.
I20260808 05:00:50.784606 26116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } }
I20260808 05:00:50.784847 26116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 041f2c1fa7604f71b90077dde309c122, State: Initialized, Role: FOLLOWER
I20260808 05:00:50.785007 26116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } }
I20260808 05:00:50.785101 26116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:00:50.785156 26116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:00:50.785216 26116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 1 FOLLOWER]: Advancing to term 2
I20260808 05:00:50.786221 26116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } }
I20260808 05:00:50.786321 26116 leader_election.cc:302] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 041f2c1fa7604f71b90077dde309c122; no voters:
I20260808 05:00:50.786487 26116 leader_election.cc:288] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 05:00:50.786550 26129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 05:00:50.786736 26116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:00:50.786773 26129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [term 2 LEADER]: Becoming Leader. State: Replica: 041f2c1fa7604f71b90077dde309c122, State: Running, Role: LEADER
I20260808 05:00:50.786898 26129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } }
I20260808 05:00:50.787256 26129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 041f2c1fa7604f71b90077dde309c122. Latest consensus state: current_term: 2 leader_uuid: "041f2c1fa7604f71b90077dde309c122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } } }
I20260808 05:00:50.787340 26129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:50.787497 26130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "041f2c1fa7604f71b90077dde309c122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "041f2c1fa7604f71b90077dde309c122" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45379 } } }
I20260808 05:00:50.787583 26130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:50.787799 26144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:00:50.788281 26144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:00:50.788426 26144 catalog_manager.cc:1259] Loaded cluster ID: 2ebfed64b10c4713b7328874ef12bd02
I20260808 05:00:50.788503 26144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:00:50.788717 26116 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 05:00:50.792438 26144 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:00:50.793011 26144 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Loaded TSK: 0
I20260808 05:00:50.793357 26144 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 05:00:50.884920 26124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:50.885106 26124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:50.885136 26124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:50.886300 26124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:50.886341 26124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:50.886376 26124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:00:50.887976 26124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.174.148:50227/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:33265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=33957
--tserver_master_addrs=127.3.174.190:45379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:50.888206 26124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:50.888449 26124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:50.896831 26194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:50.896901 26190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:50.897058 26191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:50.897238 26124 server_base.cc:1034] running on GCE node
I20260808 05:00:50.897542 26124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:50.897825 26124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:50.898972 26124 hybrid_clock.cc:648] HybridClock initialized: now 1786165250898949 us; error 35 us; skew 500 ppm
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:50.902384 26124 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:00:50.902977 26124 webserver.cc:460] Webserver started at http://127.3.174.129:33957/ using document root <none> and password file <none>
I20260808 05:00:50.903323 26124 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:50.903542 26124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:50.920531 26124 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 05:00:50.921772 26209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.922361 26124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:00:50.922510 26124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "a47bf364fb4b42369fc72335207ebf55"
format_stamp: "Formatted at 2026-08-08 05:00:49 on dist-test-slave-gjw7"
server_key: "a727b9279da06424bf50c15c61c6342e"
server_key_iv: "524c57f7d73cdbe840ff56a4f65eae81"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:50.923022 26124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:50.945452 26124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:50.947979 26124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:50.948194 26124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:50.948511 26124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:50.948959 26124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:50.949010 26124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.949044 26124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:50.949070 26124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:50.956525 26124 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:33265
I20260808 05:00:50.956609 26402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:33265 every 8 connection(s)
I20260808 05:00:50.957119 26124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 05:00:50 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165250, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:50.966511 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 26124
I20260808 05:00:50.966814 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:50227/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:44441
--local_ip_for_outbound_sockets=127.3.174.130
--tserver_master_addrs=127.3.174.190:45379
--webserver_port=44101
--webserver_interface=127.3.174.130
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.174.148:50227/kms with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:00:50.968916 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:50.958052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:40823 (local address 127.3.174.190:45379)
0808 05:00:50.958272 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:50.958280 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:50.958312 (+ 32us) server_negotiation.cc:407] Connection header received
0808 05:00:50.958949 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:50.958954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:50.959001 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:50.959107 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:50.959924 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.961043 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.962499 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.962911 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.962995 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:50.965221 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:50.965225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:50.965225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:50.965237 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:50.967037 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:50.967482 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:50.967485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:50.967486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:50.967606 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:50.967882 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:50.967883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:50.967883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:50.968182 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:50.968270 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:50.968628 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 05:00:50.968748 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260808 05:00:50.968926 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:50.957885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45379 (local address 127.3.174.129:40823)
0808 05:00:50.958054 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:50.958102 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:50.958719 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:50.959189 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:50.959194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:50.959413 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:50.959779 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:50.959786 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.961155 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.961156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:50.962406 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:50.962410 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:50.963026 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:50.963027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:50.963164 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:50.963638 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:50.963649 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:50.965094 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:50.967124 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:50.967131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:50.967133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:50.967392 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:50.967679 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:50.967680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:50.967680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:50.967814 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:50.968254 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:50.968260 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:50.968419 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 05:00:50.968530 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 05:00:50.968681 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260808 05:00:50.969718 26403 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45379
I20260808 05:00:50.969820 26403 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:50.969996 26403 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
W20260808 05:00:50.970991 26403 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:45379 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 05:00:50.971151 26403 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:50.971266 26403 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
I20260808 05:00:50.971719 26403 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:50.971813 26403 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
W20260808 05:00:50.972167 26403 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 05:00:51.076498 26410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:51.076773 26410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:51.076812 26410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:51.077957 26410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:51.077998 26410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:51.078023 26410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:00:51.079864 26410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.174.148:50227/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:44441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=44101
--tserver_master_addrs=127.3.174.190:45379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:51.080080 26410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:51.080341 26410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:51.086409 26418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:51.086679 26421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:51.087028 26417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:51.087630 26410 server_base.cc:1034] running on GCE node
I20260808 05:00:51.087886 26410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:51.088182 26410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:51.089358 26410 hybrid_clock.cc:648] HybridClock initialized: now 1786165251089332 us; error 41 us; skew 500 ppm
Aug 08 05:00:51 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:51.092509 26410 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:00:51.093017 26410 webserver.cc:460] Webserver started at http://127.3.174.130:44101/ using document root <none> and password file <none>
I20260808 05:00:51.093348 26410 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:51.093429 26410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 05:00:51 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:51.110773 26410 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:00:51.111884 26435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:51.112120 26410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 05:00:51.112211 26410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c"
format_stamp: "Formatted at 2026-08-08 05:00:50 on dist-test-slave-gjw7"
server_key: "a96a54e04946c6d637446c14c294a8eb"
server_key_iv: "2e1efa05f79e611b72a11f71485233f5"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:51.112748 26410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:51.120713 26410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:51.122718 26410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:51.122854 26410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:51.123071 26410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:51.123410 26410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:51.123448 26410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:51.123474 26410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:51.123514 26410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:51.130455 26410 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:44441
I20260808 05:00:51.130854 26410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 05:00:51.131392 26647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:44441 every 8 connection(s)
I20260808 05:00:51.134399 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 26410
I20260808 05:00:51.134658 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.174.148:50227/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:33965
--local_ip_for_outbound_sockets=127.3.174.131
--tserver_master_addrs=127.3.174.190:45379
--webserver_port=43047
--webserver_interface=127.3.174.131
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.174.148:50227/kms with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 05:00:51 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:51.140856 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:51.131719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45379 (local address 127.3.174.130:33915)
0808 05:00:51.131881 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:51.131915 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:51.132411 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:51.132657 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:51.132661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:51.132883 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:51.133071 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:51.133076 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.133788 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.133789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:51.134787 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:51.134790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.135262 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.135263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:51.135401 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:51.135859 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:51.135870 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:51.137584 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:51.139279 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:51.139286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:51.139288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:51.139540 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:51.139744 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:51.139745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:51.139745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:51.139886 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:51.140182 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:51.140188 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:51.140359 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 05:00:51.140466 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 05:00:51.140615 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260808 05:00:51.141568 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:51.131687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:33915 (local address 127.3.174.190:45379)
0808 05:00:51.131739 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:51.131742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:51.131993 (+ 251us) server_negotiation.cc:407] Connection header received
0808 05:00:51.132519 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:51.132521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:51.132533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:51.132585 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:51.133161 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.133700 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.134881 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.135115 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.135156 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:51.137666 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:51.137668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:51.137668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:51.137676 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:51.139182 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:51.139615 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:51.139616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:51.139617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:51.139676 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:51.139959 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:51.139960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:51.139960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:51.140096 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:51.140154 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:51.141389 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0808 05:00:51.141478 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 05:00:51.141947 26648 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45379
I20260808 05:00:51.142089 26648 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:51.142293 26648 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
W20260808 05:00:51.143265 26648 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:45379 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 05:00:51.143344 26648 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:51.143453 26648 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
I20260808 05:00:51.143846 26648 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:51.143959 26648 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
W20260808 05:00:51.144392 26648 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 05:00:51.233695 26653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:51.233922 26653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:51.233954 26653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:51.235095 26653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:51.235136 26653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:51.235170 26653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:00:51.237226 26653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.174.148:50227/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:33965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=43047
--tserver_master_addrs=127.3.174.190:45379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:51.237637 26653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:51.237920 26653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:51.243080 26668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:51.243428 26664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:51.243816 26670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:51.246461 26653 server_base.cc:1034] running on GCE node
I20260808 05:00:51.246716 26653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:51.246978 26653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:51.248188 26653 hybrid_clock.cc:648] HybridClock initialized: now 1786165251248166 us; error 35 us; skew 500 ppm
Aug 08 05:00:51 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:51.250861 26653 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:00:51.251358 26653 webserver.cc:460] Webserver started at http://127.3.174.131:43047/ using document root <none> and password file <none>
I20260808 05:00:51.251664 26653 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:51.251739 26653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 05:00:51 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:51.271535 26653 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 05:00:51.281432 26690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:51.281612 26653 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.000s
I20260808 05:00:51.281706 26653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "5aa719ea0c414a599711ebeca2f98ff6"
format_stamp: "Formatted at 2026-08-08 05:00:50 on dist-test-slave-gjw7"
server_key: "83bb35fabbaf7df5f7f77bb23b21dde7"
server_key_iv: "779ef91a5187d5b749d2016635c888bc"
server_key_version: "kuduclusterkey@0"
I20260808 05:00:51.282951 26653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:51.299288 26653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:51.301378 26653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:51.301507 26653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:51.301725 26653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:51.302105 26653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:51.302153 26653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:51.302184 26653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:51.302223 26653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:51.308806 26653 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:33965
I20260808 05:00:51.309103 26892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:33965 every 8 connection(s)
I20260808 05:00:51.309391 26653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:00:51.312301 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 26653
Aug 08 05:00:51 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:51.320863 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:51.311189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:45635 (local address 127.3.174.190:45379)
0808 05:00:51.311242 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:51.311244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:51.311954 (+ 710us) server_negotiation.cc:407] Connection header received
0808 05:00:51.312692 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:51.312694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:51.312707 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:51.312767 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:51.313535 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.314127 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.315370 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.315675 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.315752 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:51.317987 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:51.317989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:51.317989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:51.317995 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:51.319543 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:51.319992 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:51.319993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:51.319994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:51.320051 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:51.320269 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:51.320270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:51.320270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:51.320377 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:51.320433 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:51.320690 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 05:00:51.320773 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 05:00:51.321020 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:51.311192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45379 (local address 127.3.174.131:45635)
0808 05:00:51.311807 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:51.311858 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:51.312535 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:51.312851 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:51.312856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:51.313092 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:51.313420 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:51.313427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.314219 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.314220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:51.315277 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:51.315280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.315776 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.315777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:51.315930 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:51.316423 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:51.316431 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:51.317882 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:51.319667 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:51.319683 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:51.319686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:51.319925 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:51.320102 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:51.320103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:51.320103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:51.320213 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:51.320431 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:51.320437 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:51.320571 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 05:00:51.320660 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 05:00:51.320806 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":79,"threads_started":1}
I20260808 05:00:51.321499 26893 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45379
I20260808 05:00:51.321604 26893 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:51.321794 26893 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
W20260808 05:00:51.322435 26893 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:45379 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 05:00:51.322504 26893 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:51.322559 26893 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
I20260808 05:00:51.322576 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:51.312804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.3.174.190:45379)
0808 05:00:51.312915 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:51.312916 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:51.312986 (+ 70us) server_negotiation.cc:407] Connection header received
0808 05:00:51.313057 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:51.313058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:51.313069 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:51.313126 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:51.313838 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.314550 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.315554 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:51.315847 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:51.315926 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:51.317828 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:51.317830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:51.317830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:51.317838 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:51.319678 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:51.320010 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:51.320012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:51.320012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:51.320078 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:51.321974 (+ 1896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:51.321975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:51.321975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:51.322106 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:51.322277 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:51.322400 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 05:00:51.322480 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260808 05:00:51.323052 26893 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:51.323120 26893 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
W20260808 05:00:51.323385 26893 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 05:00:51 dist-test-slave-gjw7 krb5kdc[24296](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786165251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:51.972359 26403 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:51.972512 26403 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
I20260808 05:00:51.972945 26056 ts_manager.cc:194] Registered new tserver with Master: a47bf364fb4b42369fc72335207ebf55 (127.3.174.129:33265)
I20260808 05:00:51.973971 26056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:40823
I20260808 05:00:52.144616 26648 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:52.144774 26648 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
I20260808 05:00:52.145069 26056 ts_manager.cc:194] Registered new tserver with Master: 8da51ccd71764ec19ebfc1c5c39fcf8c (127.3.174.130:44441)
I20260808 05:00:52.145665 26056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:33915
Aug 08 05:00:52 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:52.323565 26893 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:52.323727 26893 heartbeater.cc:511] Master 127.3.174.190:45379 requested a full tablet report, sending...
I20260808 05:00:52.324154 26056 ts_manager.cc:194] Registered new tserver with Master: 5aa719ea0c414a599711ebeca2f98ff6 (127.3.174.131:33965)
I20260808 05:00:52.324688 26056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:45635
I20260808 05:00:52.326220 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:00:52.338385 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.327253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.3.174.190:45379)
0808 05:00:52.328268 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:52.328270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:52.328429 (+ 159us) server_negotiation.cc:407] Connection header received
0808 05:00:52.328451 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:52.328452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:52.328462 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:52.328511 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:52.329165 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.329918 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.330873 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.331146 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.331196 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.335283 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:52.335284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:52.335285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:52.335293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:52.336941 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:52.337326 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:52.337328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:52.337328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:52.337392 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:52.337566 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:52.337567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:52.337567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:52.337688 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:52.337770 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:52.338201 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 05:00:52.338285 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":990,"thread_start_us":68,"threads_started":1}
I20260808 05:00:52.340497 26056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:00:52.342842 26056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:00:52.355777 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.346074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33265 (local address 127.0.0.1:57116)
0808 05:00:52.347231 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:52.347274 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:52.347337 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:52.347697 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:52.347701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:52.347702 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:52.347762 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.347763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.349271 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.349272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.350599 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.350602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.355480 (+ 4878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.355481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.355561 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.355575 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 05:00:52.355633 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 05:00:52.355689 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"thread_start_us":87,"threads_started":1}
I20260808 05:00:52.358588 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.346549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.3.174.129:33265)
0808 05:00:52.347461 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:52.347465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:52.347474 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:00:52.347509 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:52.347510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:52.347526 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:52.347591 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:52.347835 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.349186 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.351523 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.352245 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.352315 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.352377 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:52.358463 (+ 6086us) server_negotiation.cc:299] Negotiation successful
0808 05:00:52.358511 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":72,"threads_started":1}
I20260808 05:00:52.359449 26267 tablet_service.cc:1467] Processing CreateTablet for tablet bc90c8751e9c40c1ab17ca6ee074b99a (DEFAULT_TABLE table=test-table [id=4de36319b26b4fa8979f885d341f4121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:00:52.359848 26267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc90c8751e9c40c1ab17ca6ee074b99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:52.361869 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.346299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:44441 (local address 127.0.0.1:53980)
0808 05:00:52.350879 (+ 4580us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:52.350886 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:52.352405 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:52.354299 (+ 1894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:52.354299 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:52.354301 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:52.354345 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.354346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.356133 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.356134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.359195 (+ 3061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.359197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.361640 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.361641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.361724 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.361727 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:00:52.361765 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 05:00:52.361800 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4567,"thread_start_us":91,"threads_started":1}
I20260808 05:00:52.362119 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.346396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53980 (local address 127.3.174.130:44441)
0808 05:00:52.347157 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:52.347161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:52.351184 (+ 4023us) server_negotiation.cc:407] Connection header received
0808 05:00:52.352525 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:52.352528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:52.352546 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:52.352608 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:52.354419 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.355389 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.360334 (+ 4945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.361129 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.361177 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.361582 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:52.362001 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 05:00:52.362044 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":53,"threads_started":1}
I20260808 05:00:52.362578 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.346342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43976 (local address 127.3.174.131:33965)
0808 05:00:52.347057 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:52.347062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:52.350783 (+ 3721us) server_negotiation.cc:407] Connection header received
0808 05:00:52.351258 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:52.351261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:52.351280 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:52.351332 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:52.352722 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.354026 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.359294 (+ 5268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.360022 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.360076 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.360127 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:52.362456 (+ 2329us) server_negotiation.cc:299] Negotiation successful
0808 05:00:52.362503 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":43,"threads_started":1}
I20260808 05:00:52.362769 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.346076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:33965 (local address 127.0.0.1:43976)
0808 05:00:52.350726 (+ 4650us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:52.350736 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:52.350814 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:52.351429 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:52.351429 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:52.351431 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:52.351466 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.351467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.354106 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.354106 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.358253 (+ 4147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.358256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.360189 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.360190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.360246 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.360249 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:00:52.362656 (+ 2407us) client_negotiation.cc:240] Negotiation successful
0808 05:00:52.362709 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4633,"thread_start_us":80,"threads_started":1}
I20260808 05:00:52.362792 26487 tablet_service.cc:1467] Processing CreateTablet for tablet bc90c8751e9c40c1ab17ca6ee074b99a (DEFAULT_TABLE table=test-table [id=4de36319b26b4fa8979f885d341f4121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:00:52.363157 26487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc90c8751e9c40c1ab17ca6ee074b99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:52.363454 26773 tablet_service.cc:1467] Processing CreateTablet for tablet bc90c8751e9c40c1ab17ca6ee074b99a (DEFAULT_TABLE table=test-table [id=4de36319b26b4fa8979f885d341f4121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:00:52.363739 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc90c8751e9c40c1ab17ca6ee074b99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:52.369465 26951 tablet_bootstrap.cc:492] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: Bootstrap starting.
I20260808 05:00:52.370208 26951 tablet_bootstrap.cc:654] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:52.370604 26951 log.cc:824] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:52.371030 26950 tablet_bootstrap.cc:492] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: Bootstrap starting.
I20260808 05:00:52.371763 26950 tablet_bootstrap.cc:654] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:52.372157 26950 log.cc:824] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:52.372543 26952 tablet_bootstrap.cc:492] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55: Bootstrap starting.
I20260808 05:00:52.373483 26952 tablet_bootstrap.cc:654] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:52.373862 26952 log.cc:824] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:52.377969 26951 tablet_bootstrap.cc:492] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: No bootstrap required, opened a new log
I20260808 05:00:52.378041 26951 ts_tablet_manager.cc:1397] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 05:00:52.380432 26951 raft_consensus.cc:348] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } }
I20260808 05:00:52.381145 26951 raft_consensus.cc:374] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:52.381238 26951 raft_consensus.cc:729] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa719ea0c414a599711ebeca2f98ff6, State: Initialized, Role: FOLLOWER
I20260808 05:00:52.381462 26951 consensus_queue.cc:260] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } }
I20260808 05:00:52.381826 26951 ts_tablet_manager.cc:1428] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 05:00:52.381982 26893 heartbeater.cc:503] Master 127.3.174.190:45379 was elected leader, sending a full tablet report...
I20260808 05:00:52.383180 26950 tablet_bootstrap.cc:492] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: No bootstrap required, opened a new log
I20260808 05:00:52.383249 26950 ts_tablet_manager.cc:1397] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260808 05:00:52.385818 26950 raft_consensus.cc:348] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } }
I20260808 05:00:52.386025 26950 raft_consensus.cc:374] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:52.386102 26950 raft_consensus.cc:729] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da51ccd71764ec19ebfc1c5c39fcf8c, State: Initialized, Role: FOLLOWER
I20260808 05:00:52.386279 26950 consensus_queue.cc:260] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } }
I20260808 05:00:52.386626 26950 ts_tablet_manager.cc:1428] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 05:00:52.386782 26648 heartbeater.cc:503] Master 127.3.174.190:45379 was elected leader, sending a full tablet report...
I20260808 05:00:52.388284 26952 tablet_bootstrap.cc:492] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55: No bootstrap required, opened a new log
I20260808 05:00:52.388352 26952 ts_tablet_manager.cc:1397] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260808 05:00:52.391265 26952 raft_consensus.cc:348] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } }
I20260808 05:00:52.391479 26952 raft_consensus.cc:374] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:52.391561 26952 raft_consensus.cc:729] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a47bf364fb4b42369fc72335207ebf55, State: Initialized, Role: FOLLOWER
I20260808 05:00:52.391754 26952 consensus_queue.cc:260] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } }
I20260808 05:00:52.392081 26952 ts_tablet_manager.cc:1428] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260808 05:00:52.392234 26403 heartbeater.cc:503] Master 127.3.174.190:45379 was elected leader, sending a full tablet report...
W20260808 05:00:52.458396 26404 tablet.cc:2369] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 05:00:52 dist-test-slave-gjw7 krb5kdc[24296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for HTTP/127.3.174.148@KRBTEST.COM
I20260808 05:00:52.482834 26965 raft_consensus.cc:482] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:00:52.482967 26965 raft_consensus.cc:504] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } }
I20260808 05:00:52.483402 26965 leader_election.cc:288] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a47bf364fb4b42369fc72335207ebf55 (127.3.174.129:33265), 5aa719ea0c414a599711ebeca2f98ff6 (127.3.174.131:33965)
I20260808 05:00:52.493676 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.484066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:37269 (local address 127.3.174.129:33265)
0808 05:00:52.484118 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:52.484120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:52.487112 (+ 2992us) server_negotiation.cc:407] Connection header received
0808 05:00:52.487225 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:52.487226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:52.487237 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:52.487266 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:52.487547 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.488284 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.492655 (+ 4371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.493245 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.493315 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.493326 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:52.493554 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 05:00:52.493600 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 05:00:52.494143 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.483608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33265 (local address 127.3.174.130:37269)
0808 05:00:52.487033 (+ 3425us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:52.487046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:52.487147 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:52.487379 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:52.487380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:52.487383 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:52.487464 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.487466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.488816 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.488817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.492552 (+ 3735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.492556 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.493408 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.493409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.493481 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.493484 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:00:52.493993 (+ 509us) client_negotiation.cc:240] Negotiation successful
0808 05:00:52.494041 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3400,"thread_start_us":121,"threads_started":1}
I20260808 05:00:52.495328 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc90c8751e9c40c1ab17ca6ee074b99a" candidate_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47bf364fb4b42369fc72335207ebf55" is_pre_election: true
I20260808 05:00:52.495517 26331 raft_consensus.cc:2454] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da51ccd71764ec19ebfc1c5c39fcf8c in term 0.
I20260808 05:00:52.495885 26441 leader_election.cc:302] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da51ccd71764ec19ebfc1c5c39fcf8c, a47bf364fb4b42369fc72335207ebf55; no voters:
I20260808 05:00:52.496047 26965 raft_consensus.cc:2781] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:00:52.496091 26965 raft_consensus.cc:482] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:00:52.496111 26965 raft_consensus.cc:3037] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:52.497088 26965 raft_consensus.cc:504] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } }
I20260808 05:00:52.497224 26965 leader_election.cc:288] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [CANDIDATE]: Term 1 election: Requested vote from peers a47bf364fb4b42369fc72335207ebf55 (127.3.174.129:33265), 5aa719ea0c414a599711ebeca2f98ff6 (127.3.174.131:33965)
I20260808 05:00:52.497597 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc90c8751e9c40c1ab17ca6ee074b99a" candidate_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47bf364fb4b42369fc72335207ebf55"
I20260808 05:00:52.497675 26331 raft_consensus.cc:3037] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:52.498536 26331 raft_consensus.cc:2454] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da51ccd71764ec19ebfc1c5c39fcf8c in term 1.
I20260808 05:00:52.498780 26441 leader_election.cc:302] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da51ccd71764ec19ebfc1c5c39fcf8c, a47bf364fb4b42369fc72335207ebf55; no voters:
I20260808 05:00:52.498898 26965 raft_consensus.cc:2781] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:00:52.499056 26965 raft_consensus.cc:686] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 1 LEADER]: Becoming Leader. State: Replica: 8da51ccd71764ec19ebfc1c5c39fcf8c, State: Running, Role: LEADER
I20260808 05:00:52.499145 26965 consensus_queue.cc:237] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } }
I20260808 05:00:52.499984 26056 catalog_manager.cc:5495] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da51ccd71764ec19ebfc1c5c39fcf8c (127.3.174.130). New cstate: current_term: 1 leader_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 44441 } health_report { overall_health: HEALTHY } } }
I20260808 05:00:52.501205 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.484148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:40255 (local address 127.3.174.131:33965)
0808 05:00:52.484189 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:52.484191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:52.489441 (+ 5250us) server_negotiation.cc:407] Connection header received
0808 05:00:52.489541 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:52.489542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:52.489552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:52.489580 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:00:52.489810 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.490482 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.494368 (+ 3886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.494936 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.494983 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.494994 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:52.501078 (+ 6084us) server_negotiation.cc:299] Negotiation successful
0808 05:00:52.501135 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 05:00:52.501668 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.483857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:33965 (local address 127.3.174.130:40255)
0808 05:00:52.489365 (+ 5508us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:52.489377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:52.489470 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:52.489641 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:52.489642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:52.489643 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:00:52.489730 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.489732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.490583 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.490584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.491775 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:52.491778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.500867 (+ 9089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.500868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:52.500943 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.500947 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:00:52.501543 (+ 596us) client_negotiation.cc:240] Negotiation successful
0808 05:00:52.501589 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5487,"thread_start_us":68,"threads_started":1}
I20260808 05:00:52.502144 26808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc90c8751e9c40c1ab17ca6ee074b99a" candidate_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa719ea0c414a599711ebeca2f98ff6" is_pre_election: true
I20260808 05:00:52.502292 26808 raft_consensus.cc:2454] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da51ccd71764ec19ebfc1c5c39fcf8c in term 0.
I20260808 05:00:52.502401 26807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc90c8751e9c40c1ab17ca6ee074b99a" candidate_uuid: "8da51ccd71764ec19ebfc1c5c39fcf8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa719ea0c414a599711ebeca2f98ff6"
I20260808 05:00:52.502451 26807 raft_consensus.cc:3037] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:52.503247 26807 raft_consensus.cc:2454] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da51ccd71764ec19ebfc1c5c39fcf8c in term 1.
I20260808 05:00:52.530155 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.526928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53988 (local address 127.3.174.130:44441)
0808 05:00:52.526974 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:52.526976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:52.526983 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:00:52.527027 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:52.527027 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:52.527050 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:52.527119 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:00:52.527266 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.528076 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.529133 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:52.529470 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:52.529540 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:52.529762 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:00:52.529904 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:00:52.529986 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:52.530051 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 05:00:52.530075 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 05:00:52.532757 26807 raft_consensus.cc:1261] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [term 1 FOLLOWER]: Refusing update from remote peer 8da51ccd71764ec19ebfc1c5c39fcf8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:00:52.533094 26331 raft_consensus.cc:1261] T bc90c8751e9c40c1ab17ca6ee074b99a P a47bf364fb4b42369fc72335207ebf55 [term 1 FOLLOWER]: Refusing update from remote peer 8da51ccd71764ec19ebfc1c5c39fcf8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:00:52.539417 26965 consensus_queue.cc:1035] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa719ea0c414a599711ebeca2f98ff6" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:00:52.539618 26965 consensus_queue.cc:1035] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a47bf364fb4b42369fc72335207ebf55" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:00:52.547842 26985 mvcc.cc:205] Tried to move back new op lower bound from 7316132874370465792 to 7316132874236936192. Current Snapshot: MvccSnapshot[applied={T|T < 7316132874370465792}]
I20260808 05:00:52.550529 26979 mvcc.cc:205] Tried to move back new op lower bound from 7316132874370465792 to 7316132874236936192. Current Snapshot: MvccSnapshot[applied={T|T < 7316132874370465792 or (T in {7316132874370465792})}]
I20260808 05:00:52.551158 26982 mvcc.cc:205] Tried to move back new op lower bound from 7316132874370465792 to 7316132874236936192. Current Snapshot: MvccSnapshot[applied={T|T < 7316132874370465792}]
I20260808 05:00:52.553702 26058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:00:52.553843 26058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:00:52.554697 26058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 041f2c1fa7604f71b90077dde309c122: Sending DeleteTablet for 3 replicas of tablet bc90c8751e9c40c1ab17ca6ee074b99a
I20260808 05:00:52.555155 26773 tablet_service.cc:1514] Processing DeleteTablet for tablet bc90c8751e9c40c1ab17ca6ee074b99a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:00:52 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:43976
I20260808 05:00:52.555585 26267 tablet_service.cc:1514] Processing DeleteTablet for tablet bc90c8751e9c40c1ab17ca6ee074b99a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:00:52 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:57116
I20260808 05:00:52.556035 26487 tablet_service.cc:1514] Processing DeleteTablet for tablet bc90c8751e9c40c1ab17ca6ee074b99a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:00:52 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:53980
I20260808 05:00:52.556253 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 26124
I20260808 05:00:52.557147 27000 tablet_replica.cc:330] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: stopping tablet replica
I20260808 05:00:52.557268 27000 raft_consensus.cc:2229] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:00:52.558403 27000 raft_consensus.cc:2258] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:00:52.558568 26994 tablet_replica.cc:330] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: stopping tablet replica
I20260808 05:00:52.558668 26994 raft_consensus.cc:2229] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:00:52.558770 26994 raft_consensus.cc:2258] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:00:52.567974 27000 ts_tablet_manager.cc:1905] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:00:52.569622 26994 ts_tablet_manager.cc:1905] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:00:52.570966 26994 ts_tablet_manager.cc:1918] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:00:52.571053 26994 log.cc:1196] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/bc90c8751e9c40c1ab17ca6ee074b99a
I20260808 05:00:52.571322 26994 ts_tablet_manager.cc:1939] T bc90c8751e9c40c1ab17ca6ee074b99a P 5aa719ea0c414a599711ebeca2f98ff6: Deleting consensus metadata
I20260808 05:00:52.571784 26032 catalog_manager.cc:4841] TS 5aa719ea0c414a599711ebeca2f98ff6 (127.3.174.131:33965): tablet bc90c8751e9c40c1ab17ca6ee074b99a (table test-table [id=4de36319b26b4fa8979f885d341f4121]) successfully deleted
W20260808 05:00:52.572919 26034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:33265 (error 108)
I20260808 05:00:52.573431 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:52.573283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33265 (local address 127.0.0.1:57118)
0808 05:00:52.573355 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:52.573393 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 05:00:52.573498 27000 ts_tablet_manager.cc:1918] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:00:52.573578 27000 log.cc:1196] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/bc90c8751e9c40c1ab17ca6ee074b99a
W20260808 05:00:52.573553 26034 catalog_manager.cc:4584] TS a47bf364fb4b42369fc72335207ebf55 (127.3.174.129:33265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc90c8751e9c40c1ab17ca6ee074b99a: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33265: connect: Connection refused (error 111)
I20260808 05:00:52.573815 27000 ts_tablet_manager.cc:1939] T bc90c8751e9c40c1ab17ca6ee074b99a P 8da51ccd71764ec19ebfc1c5c39fcf8c: Deleting consensus metadata
I20260808 05:00:52.574241 26034 catalog_manager.cc:4841] TS 8da51ccd71764ec19ebfc1c5c39fcf8c (127.3.174.130:44441): tablet bc90c8751e9c40c1ab17ca6ee074b99a (table test-table [id=4de36319b26b4fa8979f885d341f4121]) successfully deleted
I20260808 05:00:52.574986 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 26410
I20260808 05:00:52.590543 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 26653
I20260808 05:00:52.602140 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 26006
I20260808 05:00:52.614475 3770 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 05:00:52.971040 3770 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 05:00:52.971139 3770 mini_ranger.cc:66] Stopping Ranger...
I20260808 05:00:53.326164 3770 mini_ranger.cc:68] Stopped Ranger
2026-08-08 05:00:53.326 UTC [24350] LOG: received smart shutdown request
2026-08-08 05:00:53.331 UTC [24350] LOG: background worker "logical replication launcher" (PID 24378) exited with exit code 1
2026-08-08 05:00:53.331 UTC [24367] LOG: shutting down
2026-08-08 05:00:53.485 UTC [24350] LOG: database system is shut down
2026-08-08T05:00:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48713 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:00:53 dist-test-slave-gjw7 krb5kdc[27019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:00:53 dist-test-slave-gjw7 krb5kdc[27019](info): set up 2 sockets
Aug 08 05:00:53 dist-test-slave-gjw7 krb5kdc[27019](info): commencing operation
krb5kdc: starting...
W20260808 05:00:54.766987 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:00:54 dist-test-slave-gjw7 krb5kdc[27019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:00:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:00:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:00:54.891924 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33795
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:33795,127.3.174.189:36529,127.3.174.188:41941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:54.979547 27040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:54.979787 27040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:54.979812 27040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:54.981014 27040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:00:54.981065 27040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:54.981078 27040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:54.981091 27040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:00:54.981101 27040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:00:54.982724 27040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:33795,127.3.174.189:36529,127.3.174.188:41941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:54.982980 27040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:54.983209 27040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:54.988109 27048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:54.988091 27050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:54.988137 27054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:54.988641 27040 server_base.cc:1034] running on GCE node
I20260808 05:00:54.988936 27040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:54.989233 27040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:54.990482 27040 hybrid_clock.cc:648] HybridClock initialized: now 1786165254990465 us; error 36 us; skew 500 ppm
Aug 08 05:00:54 dist-test-slave-gjw7 krb5kdc[27019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165254, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:54.993036 27040 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:00:54.993569 27040 webserver.cc:460] Webserver started at http://127.3.174.190:37833/ using document root <none> and password file <none>
I20260808 05:00:54.993777 27040 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:54.993825 27040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:54.993953 27040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:00:54.995069 27040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "11939784ad7a4a8da97a5d4dc0e93376"
format_stamp: "Formatted at 2026-08-08 05:00:54 on dist-test-slave-gjw7"
server_key: "c41efd6b1493d6a2c77405e965756e5b"
server_key_iv: "28a10842efc098dee73a57103e5a2780"
server_key_version: "encryptionkey@0"
I20260808 05:00:54.995438 27040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "11939784ad7a4a8da97a5d4dc0e93376"
format_stamp: "Formatted at 2026-08-08 05:00:54 on dist-test-slave-gjw7"
server_key: "c41efd6b1493d6a2c77405e965756e5b"
server_key_iv: "28a10842efc098dee73a57103e5a2780"
server_key_version: "encryptionkey@0"
I20260808 05:00:54.996783 27040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 05:00:54.997702 27063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:54.997897 27040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 05:00:54.998001 27040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "11939784ad7a4a8da97a5d4dc0e93376"
format_stamp: "Formatted at 2026-08-08 05:00:54 on dist-test-slave-gjw7"
server_key: "c41efd6b1493d6a2c77405e965756e5b"
server_key_iv: "28a10842efc098dee73a57103e5a2780"
server_key_version: "encryptionkey@0"
I20260808 05:00:54.998072 27040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:55.006108 27040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:55.008422 27040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:55.008584 27040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:55.013113 27040 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:33795
I20260808 05:00:55.013142 27155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:33795 every 8 connection(s)
I20260808 05:00:55.013593 27040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:00:55.014497 27157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:55.016106 27157 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:00:55.017740 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 27040
I20260808 05:00:55.017858 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:00:55.018055 3770 external_mini_cluster.cc:1428] Setting key ee34d7413eb9fc88ed5e2fc34f5f4471
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165254, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.174.189@KRBTEST.COM; defaulting to no policy
I20260808 05:00:55.026731 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.016963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.3.174.190:33795)
0808 05:00:55.017078 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.017084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.017135 (+ 51us) server_negotiation.cc:407] Connection header received
0808 05:00:55.017848 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.017851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.017874 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.017975 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.018584 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.019735 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.021006 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.021376 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.021460 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.023793 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.023801 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.023801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.023807 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.025234 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.025597 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.025600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.025600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.025679 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.025875 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.025876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.025876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.026098 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.026170 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.026361 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.026468 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":41,"threads_started":1}
I20260808 05:00:55.026736 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.016800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33795 (local address 127.0.0.1:33420)
0808 05:00:55.017007 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.017051 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:55.017651 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:55.017992 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:55.017996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:55.018307 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:55.018483 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.018488 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.019853 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.019854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.020913 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.020916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.021459 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.021461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.021539 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.022050 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:55.022061 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:55.023698 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:55.025325 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.025332 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.025335 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.025525 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.025740 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.025741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.025742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.025822 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.026158 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:55.026164 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:55.026229 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 05:00:55.026309 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 05:00:55.026539 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":87,"thread_start_us":71,"threads_started":1}
I20260808 05:00:55.027525 27157 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:00:55.027931 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.027814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:36529 (local address 127.0.0.1:49190)
0808 05:00:55.027858 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.027908 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 05:00:55.028066 27064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
I20260808 05:00:55.028508 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.028417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:36529 (local address 127.0.0.1:49202)
0808 05:00:55.028459 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.028485 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 05:00:55.028635 27157 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.189:36529: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab.
Entry for principal kudu/127.3.174.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab.
I20260808 05:00:55.044622 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.189:36529
--webserver_interface=127.3.174.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab
--principal=kudu/127.3.174.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:33795,127.3.174.189:36529,127.3.174.188:41941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:00:55.077877 27157 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } attempt: 1
I20260808 05:00:55.078382 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.078252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:36529 (local address 127.0.0.1:49206)
0808 05:00:55.078312 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.078348 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 05:00:55.078727 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.078643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:36529 (local address 127.0.0.1:49218)
0808 05:00:55.078683 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.078708 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 05:00:55.078842 27157 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.189:36529: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
W20260808 05:00:55.132179 27170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:55.132413 27170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:55.132436 27170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:55.133633 27170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:00:55.133678 27170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:55.133692 27170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:55.133703 27170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:00:55.133714 27170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:00:55.135353 27170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:33795,127.3.174.189:36529,127.3.174.188:41941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab
--principal=kudu/127.3.174.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.189:36529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.174.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:55.135624 27170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:55.135852 27170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:55.140692 27178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:55.140692 27176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:55.140697 27175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:55.140709 27170 server_base.cc:1034] running on GCE node
I20260808 05:00:55.141167 27170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:55.141501 27170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:55.142668 27170 hybrid_clock.cc:648] HybridClock initialized: now 1786165255142650 us; error 33 us; skew 500 ppm
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:55.145198 27170 init.cc:377] Logged in from keytab as kudu/127.3.174.189@KRBTEST.COM (short username kudu)
I20260808 05:00:55.145656 27170 webserver.cc:460] Webserver started at http://127.3.174.189:40117/ using document root <none> and password file <none>
I20260808 05:00:55.145885 27170 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:55.145936 27170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:55.146063 27170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:00:55.147011 27157 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } attempt: 2
I20260808 05:00:55.147143 27170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/data/instance:
uuid: "8eb666ec3a20413cb13821e36d9459dd"
format_stamp: "Formatted at 2026-08-08 05:00:55 on dist-test-slave-gjw7"
server_key: "5e1a41782554f33b3c3a356013cf3c32"
server_key_iv: "f782f259f508aab80dab7e988e8ec3a0"
server_key_version: "encryptionkey@0"
I20260808 05:00:55.147421 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.147276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:36529 (local address 127.0.0.1:49230)
0808 05:00:55.147343 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.147385 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 05:00:55.147559 27170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/wal/instance:
uuid: "8eb666ec3a20413cb13821e36d9459dd"
format_stamp: "Formatted at 2026-08-08 05:00:55 on dist-test-slave-gjw7"
server_key: "5e1a41782554f33b3c3a356013cf3c32"
server_key_iv: "f782f259f508aab80dab7e988e8ec3a0"
server_key_version: "encryptionkey@0"
I20260808 05:00:55.147753 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.147661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:36529 (local address 127.0.0.1:49242)
0808 05:00:55.147697 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.147729 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 05:00:55.147846 27157 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.189:36529: Network error: Client connection negotiation failed: client connection to 127.3.174.189:36529: connect: Connection refused (error 111)
I20260808 05:00:55.148931 27170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:00:55.149652 27192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:55.149818 27170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:55.149914 27170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/wal
uuid: "8eb666ec3a20413cb13821e36d9459dd"
format_stamp: "Formatted at 2026-08-08 05:00:55 on dist-test-slave-gjw7"
server_key: "5e1a41782554f33b3c3a356013cf3c32"
server_key_iv: "f782f259f508aab80dab7e988e8ec3a0"
server_key_version: "encryptionkey@0"
I20260808 05:00:55.150040 27170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:55.157976 27170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:55.159894 27170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:55.160010 27170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:55.163941 27170 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.189:36529
I20260808 05:00:55.163991 27276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.189:36529 every 8 connection(s)
I20260808 05:00:55.164350 27170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
I20260808 05:00:55.165241 27278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:55.167024 27278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:00:55.170401 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 27170
I20260808 05:00:55.170488 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-1/wal/instance
I20260808 05:00:55.170650 3770 external_mini_cluster.cc:1428] Setting key 74306b520f7ed91116101f4a39e51618
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:55.176215 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.167784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33436 (local address 127.3.174.190:33795)
0808 05:00:55.167863 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.167866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.168065 (+ 199us) server_negotiation.cc:407] Connection header received
0808 05:00:55.168555 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.168556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.168573 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.168627 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.169283 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.169802 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.170831 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.171041 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.171097 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.173419 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.173421 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.173421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.173429 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.174729 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.175175 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.175177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.175178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.175258 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.175510 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.175511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.175511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.175666 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.175748 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.176013 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.176114 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 05:00:55.176283 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.167711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33795 (local address 127.0.0.1:33436)
0808 05:00:55.167943 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.167978 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:55.168448 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:55.168709 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:55.168713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:55.168985 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:55.169187 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.169192 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.169889 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.169891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.170753 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.170757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.171112 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.171113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.171241 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.171729 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:55.171738 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:55.173327 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:55.174810 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.174818 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.174821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.175069 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.175342 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.175343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.175343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.175451 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.175736 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:55.175740 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:55.175873 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 05:00:55.175966 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 05:00:55.176103 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260808 05:00:55.177191 27278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.174.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.188@KRBTEST.COM" created.
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:00:55.185307 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.177494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:36529 (local address 127.0.0.1:49246)
0808 05:00:55.177546 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.177556 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:55.177593 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:55.177825 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:55.177826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:55.177967 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:55.178015 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.178017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.178846 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.178847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.179382 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.179384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.179803 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.179804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.179868 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.180335 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:55.180340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:55.182712 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:55.184228 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.184230 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.184230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.184446 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.184670 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.184671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.184671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.184735 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.184993 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:55.184994 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:55.185041 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 05:00:55.185111 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 05:00:55.185197 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 05:00:55.185431 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.177541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.3.174.189:36529)
0808 05:00:55.177709 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.177713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.177728 (+ 15us) server_negotiation.cc:407] Connection header received
0808 05:00:55.177745 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.177747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.177757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.177816 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.178077 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.178768 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.179444 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.179740 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.179797 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.182796 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.182806 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.182807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.182813 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.184150 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.184533 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.184537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.184537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.184603 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.184790 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.184791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.184791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.184933 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.184995 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.185147 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.185286 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260808 05:00:55.186079 27278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:00:55.186446 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.186305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51604)
0808 05:00:55.186379 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.186426 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 05:00:55.186589 27200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
I20260808 05:00:55.187005 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.186918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51616)
0808 05:00:55.186959 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.186983 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 05:00:55.187101 27278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:41941: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab.
Entry for principal kudu/127.3.174.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab.
I20260808 05:00:55.199950 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.188:41941
--webserver_interface=127.3.174.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab
--principal=kudu/127.3.174.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:33795,127.3.174.189:36529,127.3.174.188:41941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:00:55.236334 27278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } attempt: 1
I20260808 05:00:55.236860 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.236690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51628)
0808 05:00:55.236763 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.236819 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 05:00:55.237203 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.237116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51634)
0808 05:00:55.237154 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.237181 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 05:00:55.237351 27278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:41941: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
I20260808 05:00:55.238981 27157 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } attempt: 3
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165254, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:00:55.245494 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.239318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.3.174.189:36529)
0808 05:00:55.239380 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.239382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.239393 (+ 11us) server_negotiation.cc:407] Connection header received
0808 05:00:55.239441 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.239442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.239454 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.239500 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.239837 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.240326 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.240978 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.241183 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.241238 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.243155 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.243157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.243157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.243164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.244419 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.244787 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.244788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.244788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.244843 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.245020 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.245020 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.245021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.245117 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.245163 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.245361 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.245429 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 05:00:55.245523 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.239257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:36529 (local address 127.0.0.1:49250)
0808 05:00:55.239325 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.239335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:55.239387 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:55.239504 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:55.239505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:55.239666 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:55.239743 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.239745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.240408 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.240409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.240901 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.240902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.241242 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.241243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.241355 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.241769 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:55.241774 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:55.243059 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:55.244496 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.244497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.244498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.244722 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.244898 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.244899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.244899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.244967 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.245167 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:55.245168 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:55.245225 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 05:00:55.245320 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 05:00:55.245423 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 05:00:55.246022 27157 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:00:55.246368 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.246261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51646)
0808 05:00:55.246308 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.246343 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 05:00:55.246662 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.246578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51656)
0808 05:00:55.246619 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.246645 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 05:00:55.246755 27157 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:41941: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
I20260808 05:00:55.277923 27157 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } attempt: 1
I20260808 05:00:55.278486 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.278321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51668)
0808 05:00:55.278398 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.278448 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 05:00:55.278846 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.278744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51678)
0808 05:00:55.278789 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.278821 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 05:00:55.278965 27157 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:41941: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
W20260808 05:00:55.289132 27291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:55.289428 27291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:55.289466 27291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:55.290629 27291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:00:55.290673 27291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:55.290690 27291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:55.290706 27291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:00:55.290722 27291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:00:55.292310 27291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:33795,127.3.174.189:36529,127.3.174.188:41941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab
--principal=kudu/127.3.174.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.188:41941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.174.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:55.292565 27291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:55.292778 27291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 05:00:55.297717 27291 server_base.cc:1034] running on GCE node
W20260808 05:00:55.297681 27306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:55.297701 27302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:55.297688 27303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:55.298125 27291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:55.298419 27291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:55.299571 27291 hybrid_clock.cc:648] HybridClock initialized: now 1786165255299558 us; error 39 us; skew 500 ppm
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:55.302373 27291 init.cc:377] Logged in from keytab as kudu/127.3.174.188@KRBTEST.COM (short username kudu)
I20260808 05:00:55.302831 27291 webserver.cc:460] Webserver started at http://127.3.174.188:42679/ using document root <none> and password file <none>
I20260808 05:00:55.303036 27291 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:55.303084 27291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:55.303222 27291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:00:55.304316 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/data/instance:
uuid: "ee529adfec3044b298bbfbc82319f428"
format_stamp: "Formatted at 2026-08-08 05:00:55 on dist-test-slave-gjw7"
server_key: "fc84ddb17e9522f82744e07629a94938"
server_key_iv: "c1e361a1d5b6567a5ed971a1eb76c0bb"
server_key_version: "encryptionkey@0"
I20260808 05:00:55.304657 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/wal/instance:
uuid: "ee529adfec3044b298bbfbc82319f428"
format_stamp: "Formatted at 2026-08-08 05:00:55 on dist-test-slave-gjw7"
server_key: "fc84ddb17e9522f82744e07629a94938"
server_key_iv: "c1e361a1d5b6567a5ed971a1eb76c0bb"
server_key_version: "encryptionkey@0"
I20260808 05:00:55.305503 27278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } attempt: 2
I20260808 05:00:55.305917 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.305780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51686)
0808 05:00:55.305843 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.305882 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 05:00:55.306290 27291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:00:55.306435 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.306273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51694)
0808 05:00:55.306340 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.306403 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 05:00:55.306535 27278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:41941: Network error: Client connection negotiation failed: client connection to 127.3.174.188:41941: connect: Connection refused (error 111)
I20260808 05:00:55.307113 27320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:55.307289 27291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:00:55.307377 27291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/wal
uuid: "ee529adfec3044b298bbfbc82319f428"
format_stamp: "Formatted at 2026-08-08 05:00:55 on dist-test-slave-gjw7"
server_key: "fc84ddb17e9522f82744e07629a94938"
server_key_iv: "c1e361a1d5b6567a5ed971a1eb76c0bb"
server_key_version: "encryptionkey@0"
I20260808 05:00:55.307452 27291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:55.319936 27291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:55.322228 27291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:55.322353 27291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:55.326534 27291 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.188:41941
I20260808 05:00:55.326602 27408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.188:41941 every 8 connection(s)
I20260808 05:00:55.326987 27291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
I20260808 05:00:55.327961 27409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:55.329587 27409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:55.335966 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 27291
I20260808 05:00:55.336054 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786165096473501-3770-0/minicluster-data/master-2/wal/instance
I20260808 05:00:55.336205 3770 external_mini_cluster.cc:1428] Setting key d6aef79b54bf08d20d6eca5c03836312
I20260808 05:00:55.338585 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.330413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33442 (local address 127.3.174.190:33795)
0808 05:00:55.330519 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.330523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.330699 (+ 176us) server_negotiation.cc:407] Connection header received
0808 05:00:55.331213 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.331214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.331227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.331280 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.331922 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.332391 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.333360 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.333567 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.333625 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.335656 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.335658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.335658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.335666 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.337224 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.337659 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.337660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.337660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.337713 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.337962 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.337962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.337962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.338068 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.338116 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.338451 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.338519 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 05:00:55.338762 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.330309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33795 (local address 127.0.0.1:33442)
0808 05:00:55.330575 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.330617 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:55.331115 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:55.331369 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:55.331373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:55.331635 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:55.331829 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.331833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.332472 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.332473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.333252 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.333255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.333639 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.333640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.333760 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.334217 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:55.334226 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:55.335568 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:55.337340 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.337346 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.337348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.337583 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.337770 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.337770 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.337771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.337901 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.338122 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:55.338126 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:55.338276 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 05:00:55.338390 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 05:00:55.338539 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":115,"threads_started":1}
I20260808 05:00:55.339309 27409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:00:55.345013 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.336559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51704 (local address 127.3.174.188:41941)
0808 05:00:55.336707 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.336711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.336764 (+ 53us) server_negotiation.cc:407] Connection header received
0808 05:00:55.336835 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.336837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.336848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.336911 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.337610 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.338590 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.339521 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.339885 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.339938 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.342358 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.342368 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.342368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.342376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.343932 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.344258 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.344261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.344262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.344316 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.344484 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.344485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.344486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.344625 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.344765 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.344836 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.344910 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260808 05:00:55.345656 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.339554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:36529 (local address 127.0.0.1:49262)
0808 05:00:55.339599 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.339609 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:55.339640 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:55.339787 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:55.339788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:55.339911 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:55.339975 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.339977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.340740 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.340742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.341483 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.341486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.341815 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.341816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.341881 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.342250 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:55.342254 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:55.343335 (+ 1081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:55.344714 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.344715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.344716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.344871 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.345061 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.345062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.345062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.345120 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.345394 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:55.345395 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:55.345431 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 05:00:55.345487 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 05:00:55.345576 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 05:00:55.345676 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.339636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.3.174.189:36529)
0808 05:00:55.339692 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.339694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.339701 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:00:55.339725 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.339726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.339735 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.339779 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.340059 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.340664 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.341560 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.341759 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.341811 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.343420 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.343421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.343421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.343428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.344639 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.344932 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.344933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.344933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.344986 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.345177 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.345177 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.345178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.345328 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.345403 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.345524 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.345599 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 05:00:55.346045 27409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:55.352881 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.346284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51706)
0808 05:00:55.346326 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.346334 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:55.346365 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:55.346465 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:55.346465 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:55.346588 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:55.346618 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.346619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.347540 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.347541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.348303 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.348307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.348645 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.348646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.348699 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.349061 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:55.349067 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:55.350424 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:55.351969 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.351971 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.351971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.352172 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.352355 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.352356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.352356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.352421 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.352675 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:55.352676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:55.352703 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 05:00:55.352746 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 05:00:55.352813 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 05:00:55.353003 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.346321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51706 (local address 127.3.174.188:41941)
0808 05:00:55.346353 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.346355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.346363 (+ 8us) server_negotiation.cc:407] Connection header received
0808 05:00:55.346409 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.346409 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.346417 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.346459 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.346686 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.347435 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.348401 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.348583 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.348625 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.350510 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.350511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.350511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.350519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.351876 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.352245 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.352246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.352246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.352294 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.352482 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.352483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.352483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.352584 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.352658 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.352798 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.352874 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 05:00:55.353873 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.345950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.3.174.190:33795)
0808 05:00:55.345991 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.345993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.346016 (+ 23us) server_negotiation.cc:407] Connection header received
0808 05:00:55.346076 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.346077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.346085 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.346129 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.346698 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.347455 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.348399 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.348629 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.348689 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.351065 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.351066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.351067 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.351074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.352576 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.352916 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.352917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.352918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.353069 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.353314 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.353315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.353315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.353461 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.353666 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.353705 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.353777 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 05:00:55.354113 27157 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } attempt: 2
I20260808 05:00:55.354789 27409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428: Bootstrap starting.
I20260808 05:00:55.356122 27409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:55.356494 27409 log.cc:824] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:55.357466 27409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428: No bootstrap required, opened a new log
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165254, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:00:55.359828 27409 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } }
I20260808 05:00:55.360009 27409 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:55.360052 27409 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee529adfec3044b298bbfbc82319f428, State: Initialized, Role: FOLLOWER
I20260808 05:00:55.360212 27409 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } }
I20260808 05:00:55.360594 27409 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:00:55.360651 27423 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.360741 27423 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:00:55.361495 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.354333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51720)
0808 05:00:55.354468 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.354479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:55.354532 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:55.354648 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:55.354649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:55.354796 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:55.354839 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.354841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.355499 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.355500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.356262 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.356264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.356651 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.356652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.356702 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.357438 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:55.357442 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:55.358604 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:55.360073 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.360074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.360075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.360270 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.360564 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.360565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.360565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.360631 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.361176 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:55.361177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:55.361228 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 05:00:55.361326 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 05:00:55.361427 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 05:00:55.361856 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.354386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.3.174.188:41941)
0808 05:00:55.354419 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.354421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.354521 (+ 100us) server_negotiation.cc:407] Connection header received
0808 05:00:55.354575 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.354576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.354585 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.354630 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.354907 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.355429 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.356352 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.356532 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.356590 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.358683 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.358684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.358684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.358691 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.359982 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.360421 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.360422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.360422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.360487 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.360740 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.360741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.360741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.361114 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.361180 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.361689 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.361771 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
W20260808 05:00:55.362691 27443 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:00:55.362743 27443 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:00:55.362922 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.354835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.3.174.189:36529)
0808 05:00:55.354876 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.354878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.354897 (+ 19us) server_negotiation.cc:407] Connection header received
0808 05:00:55.354919 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.354920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.354928 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.354970 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.355526 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.356242 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.357093 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.357267 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.357377 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.360034 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.360035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.360035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.360043 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.361700 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.362109 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.362111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.362111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.362173 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.362360 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.362361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.362361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.362500 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.362712 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.362757 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.362842 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 05:00:55.363746 27157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376: Bootstrap starting.
I20260808 05:00:55.364579 27157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:55.364924 27157 log.cc:824] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:55.364955 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:00:55.365969 27157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376: No bootstrap required, opened a new log
I20260808 05:00:55.363579 27291 master_runner.cc:186] Error getting master registration for 127.3.174.190:33795: OK, instance_id { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" instance_seqno: 1786165255011782 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 05:00:55.366851 27291 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 05:00:55.367515 27157 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } }
I20260808 05:00:55.367617 27157 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:55.367636 27157 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11939784ad7a4a8da97a5d4dc0e93376, State: Initialized, Role: FOLLOWER
I20260808 05:00:55.367728 27157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } }
I20260808 05:00:55.367993 27157 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:00:55.367956 27452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.368029 27452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 05:00:55.370282 27469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:00:55.370349 27469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:00:55.370916 27040 master_runner.cc:186] Error getting master registration for 127.3.174.189:36529: OK, instance_id { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" instance_seqno: 1786165255162827 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 05:00:55.373948 27040 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 05:00:55.382297 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.370123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.3.174.190:33795)
0808 05:00:55.370160 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.370161 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.371058 (+ 897us) server_negotiation.cc:407] Connection header received
0808 05:00:55.371148 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.371149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.371161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.371188 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.372074 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.372774 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.377490 (+ 4716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.377811 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.377853 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.379435 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.379437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.379437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.379445 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.380980 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.381412 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.381413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.381414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.381473 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.381664 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.381665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.381666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.381793 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.381897 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.382111 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.382193 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 05:00:55.385758 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.370037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.3.174.189:36529)
0808 05:00:55.370095 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.370097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.372029 (+ 1932us) server_negotiation.cc:407] Connection header received
0808 05:00:55.374115 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.374116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.374129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.374159 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.375771 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.376500 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.379156 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.379497 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.379542 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.381493 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.381494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.381495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.381503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.383265 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.384812 (+ 1547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.384813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.384814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.384883 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.385084 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.385085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.385085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.385224 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.385367 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.385574 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.385664 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 05:00:55.388317 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.369778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51736 (local address 127.3.174.188:41941)
0808 05:00:55.369819 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.369821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.373991 (+ 4170us) server_negotiation.cc:407] Connection header received
0808 05:00:55.374987 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.374989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.375001 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.375030 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.377449 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.378138 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.379895 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.380188 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.380234 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.385437 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.385439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.385439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.385449 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.387123 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.387518 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.387519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.387520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.387581 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.387769 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.387770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.387770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.387905 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.387993 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.388134 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.388214 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 05:00:55.397663 27278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } attempt: 3
Aug 08 05:00:55 dist-test-slave-gjw7 krb5kdc[27019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
I20260808 05:00:55.406555 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.397971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:41941 (local address 127.0.0.1:51742)
0808 05:00:55.398042 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:55.398055 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:55.398100 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:55.398234 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:55.398235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:55.398446 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:55.398504 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.398506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.399423 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.399424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.400258 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:55.400261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.400715 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.400716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:55.400797 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.401828 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:55.401834 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:55.403428 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:55.405318 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.405319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.405320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.405553 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.405761 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:55.405762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:55.405762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:55.405828 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:55.406151 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:55.406153 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:55.406218 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 05:00:55.406310 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 05:00:55.406436 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 05:00:55.406922 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:55.398038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.3.174.188:41941)
0808 05:00:55.398080 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:55.398082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:55.398091 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:00:55.398154 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:55.398155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:55.398165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:55.398222 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:55.398574 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.399328 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.400344 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:55.400636 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:55.400715 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:55.403537 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:55.403539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:55.403539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:55.403548 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:55.405196 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.405631 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.405632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.405633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.405698 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:55.405885 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:55.405885 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:55.405886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:55.406042 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:55.406110 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:55.406730 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 05:00:55.406825 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 05:00:55.408711 27278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd: Bootstrap starting.
I20260808 05:00:55.409525 27278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:55.409888 27278 log.cc:824] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:55.411293 27278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd: No bootstrap required, opened a new log
I20260808 05:00:55.413391 27278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } }
I20260808 05:00:55.413545 27278 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:55.413583 27278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb666ec3a20413cb13821e36d9459dd, State: Initialized, Role: FOLLOWER
I20260808 05:00:55.413712 27278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } }
I20260808 05:00:55.414068 27278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:00:55.414059 27480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.414294 27480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [sys.catalog]: This master's current role is: FOLLOWER
W20260808 05:00:55.416059 27496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:00:55.416266 27496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:00:55.418649 27170 master_runner.cc:263] No leader master found from master 8eb666ec3a20413cb13821e36d9459dd
I20260808 05:00:55.418704 27170 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 05:00:55.509495 27075 tablet.cc:2369] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:00:55.547382 27452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:00:55.547525 27452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } }
I20260808 05:00:55.547781 27452 leader_election.cc:288] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb666ec3a20413cb13821e36d9459dd (127.3.174.189:36529), ee529adfec3044b298bbfbc82319f428 (127.3.174.188:41941)
I20260808 05:00:55.548087 27362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11939784ad7a4a8da97a5d4dc0e93376" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee529adfec3044b298bbfbc82319f428" is_pre_election: true
I20260808 05:00:55.548099 27236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11939784ad7a4a8da97a5d4dc0e93376" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb666ec3a20413cb13821e36d9459dd" is_pre_election: true
I20260808 05:00:55.548245 27362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11939784ad7a4a8da97a5d4dc0e93376 in term 0.
I20260808 05:00:55.548243 27236 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11939784ad7a4a8da97a5d4dc0e93376 in term 0.
I20260808 05:00:55.548431 27069 leader_election.cc:302] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11939784ad7a4a8da97a5d4dc0e93376, ee529adfec3044b298bbfbc82319f428; no voters:
I20260808 05:00:55.548570 27452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:00:55.548614 27452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:00:55.548635 27452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:55.549436 27452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } }
I20260808 05:00:55.549564 27452 leader_election.cc:288] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb666ec3a20413cb13821e36d9459dd (127.3.174.189:36529), ee529adfec3044b298bbfbc82319f428 (127.3.174.188:41941)
I20260808 05:00:55.549752 27236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11939784ad7a4a8da97a5d4dc0e93376" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb666ec3a20413cb13821e36d9459dd"
I20260808 05:00:55.549753 27362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11939784ad7a4a8da97a5d4dc0e93376" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee529adfec3044b298bbfbc82319f428"
I20260808 05:00:55.549834 27236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:55.549834 27362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:55.550693 27362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11939784ad7a4a8da97a5d4dc0e93376 in term 1.
I20260808 05:00:55.550693 27236 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11939784ad7a4a8da97a5d4dc0e93376 in term 1.
I20260808 05:00:55.550876 27069 leader_election.cc:302] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11939784ad7a4a8da97a5d4dc0e93376, ee529adfec3044b298bbfbc82319f428; no voters:
I20260808 05:00:55.550968 27452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:00:55.551077 27452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [term 1 LEADER]: Becoming Leader. State: Replica: 11939784ad7a4a8da97a5d4dc0e93376, State: Running, Role: LEADER
I20260808 05:00:55.551170 27452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } }
I20260808 05:00:55.551633 27498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11939784ad7a4a8da97a5d4dc0e93376. Latest consensus state: current_term: 1 leader_uuid: "11939784ad7a4a8da97a5d4dc0e93376" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.551717 27498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:55.551942 27500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:00:55.552124 27500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:00:55.553516 27362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [term 1 FOLLOWER]: Refusing update from remote peer 11939784ad7a4a8da97a5d4dc0e93376: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:00:55.553871 27236 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [term 1 FOLLOWER]: Refusing update from remote peer 11939784ad7a4a8da97a5d4dc0e93376: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:00:55.553921 27498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:00:55.554222 27498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:00:55.555286 27423 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11939784ad7a4a8da97a5d4dc0e93376. Latest consensus state: current_term: 1 leader_uuid: "11939784ad7a4a8da97a5d4dc0e93376" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.555399 27423 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:00:55.555346 27480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11939784ad7a4a8da97a5d4dc0e93376. Latest consensus state: current_term: 1 leader_uuid: "11939784ad7a4a8da97a5d4dc0e93376" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.555434 27480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:00:55.556583 27452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11939784ad7a4a8da97a5d4dc0e93376" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.556666 27452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:55.556792 27423 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11939784ad7a4a8da97a5d4dc0e93376" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.556870 27423 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:00:55.557029 27501 mvcc.cc:205] Tried to move back new op lower bound from 7316132886745280512 to 7316132886738391040. Current Snapshot: MvccSnapshot[applied={T|T < 7316132886745280512}]
I20260808 05:00:55.557475 27502 mvcc.cc:205] Tried to move back new op lower bound from 7316132886745280512 to 7316132886738391040. Current Snapshot: MvccSnapshot[applied={T|T < 7316132886745280512}]
I20260808 05:00:55.557512 27498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11939784ad7a4a8da97a5d4dc0e93376" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.557580 27498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:55.558041 27480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11939784ad7a4a8da97a5d4dc0e93376" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11939784ad7a4a8da97a5d4dc0e93376" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33795 } } peers { permanent_uuid: "8eb666ec3a20413cb13821e36d9459dd" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 36529 } } peers { permanent_uuid: "ee529adfec3044b298bbfbc82319f428" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 41941 } } }
I20260808 05:00:55.558125 27480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:00:55.558298 27500 catalog_manager.cc:1347] Generated new cluster ID: 2a591b518acb4f6ba7ca1cce478ca118
I20260808 05:00:55.558341 27500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:00:55.569130 27500 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:00:55.569823 27500 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:00:55.582716 27500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11939784ad7a4a8da97a5d4dc0e93376: Generated new TSK 0
I20260808 05:00:55.583029 27500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:00:56.363375 27443 catalog_manager.cc:1259] Loaded cluster ID: 2a591b518acb4f6ba7ca1cce478ca118
I20260808 05:00:56.363456 27443 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428: loading cluster ID for follower catalog manager: success
I20260808 05:00:56.364244 27443 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428: acquiring CA information for follower catalog manager: success
I20260808 05:00:56.364506 27443 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee529adfec3044b298bbfbc82319f428: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 05:00:56.416802 27496 catalog_manager.cc:1259] Loaded cluster ID: 2a591b518acb4f6ba7ca1cce478ca118
I20260808 05:00:56.416885 27496 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd: loading cluster ID for follower catalog manager: success
I20260808 05:00:56.417704 27496 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd: acquiring CA information for follower catalog manager: success
I20260808 05:00:56.418040 27496 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8eb666ec3a20413cb13821e36d9459dd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 05:00:56.647367 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 27040
I20260808 05:00:56.663290 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 27170
I20260808 05:00:56.679432 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 27291
2026-08-08T05:00:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3105 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:00:56 dist-test-slave-gjw7 krb5kdc[27525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:00:56 dist-test-slave-gjw7 krb5kdc[27525](info): set up 2 sockets
Aug 08 05:00:56 dist-test-slave-gjw7 krb5kdc[27525](info): commencing operation
krb5kdc: starting...
W20260808 05:00:57.867038 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:00:57 dist-test-slave-gjw7 krb5kdc[27525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:00:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:00:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:00:57.985715 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46829
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:41475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:46829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:58.068328 27548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:58.068526 27548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:58.068547 27548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:58.069681 27548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:00:58.069721 27548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:58.069734 27548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:58.069746 27548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:00:58.069761 27548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:00:58.071206 27548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:41475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:46829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:58.071411 27548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:58.071588 27548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 05:00:58.076427 27548 server_base.cc:1034] running on GCE node
W20260808 05:00:58.076399 27558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:58.076392 27562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:58.076406 27559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:58.076817 27548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:58.077078 27548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:58.078241 27548 hybrid_clock.cc:648] HybridClock initialized: now 1786165258078226 us; error 33 us; skew 500 ppm
Aug 08 05:00:58 dist-test-slave-gjw7 krb5kdc[27525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165258, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:58.086591 27548 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:00:58.087062 27548 webserver.cc:460] Webserver started at http://127.3.174.190:43073/ using document root <none> and password file <none>
I20260808 05:00:58.087296 27548 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:58.087347 27548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:58.087476 27548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:00:58.088562 27548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "2873b39d66874f72aed190c6de4cdbeb"
format_stamp: "Formatted at 2026-08-08 05:00:58 on dist-test-slave-gjw7"
server_key: "e7752e22a5f60729cd41085a5b0ac5e6"
server_key_iv: "78bc35f57eff05e45fa979821e7017a9"
server_key_version: "encryptionkey@0"
I20260808 05:00:58.088891 27548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "2873b39d66874f72aed190c6de4cdbeb"
format_stamp: "Formatted at 2026-08-08 05:00:58 on dist-test-slave-gjw7"
server_key: "e7752e22a5f60729cd41085a5b0ac5e6"
server_key_iv: "78bc35f57eff05e45fa979821e7017a9"
server_key_version: "encryptionkey@0"
I20260808 05:00:58.090246 27548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:00:58.090907 27578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:58.091066 27548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:58.091153 27548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "2873b39d66874f72aed190c6de4cdbeb"
format_stamp: "Formatted at 2026-08-08 05:00:58 on dist-test-slave-gjw7"
server_key: "e7752e22a5f60729cd41085a5b0ac5e6"
server_key_iv: "78bc35f57eff05e45fa979821e7017a9"
server_key_version: "encryptionkey@0"
I20260808 05:00:58.091246 27548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:58.101061 27548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:58.102797 27548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:58.102929 27548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:58.106905 27548 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:46829
I20260808 05:00:58.106959 27672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:46829 every 8 connection(s)
I20260808 05:00:58.107298 27548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:00:58.108114 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:58.109964 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 27548
I20260808 05:00:58.110055 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:00:58.110268 3770 external_mini_cluster.cc:1428] Setting key cd5f04088fdc2d03e76b22707120efcc
I20260808 05:00:58.110638 27673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb: Bootstrap starting.
I20260808 05:00:58.111275 27673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:58.111550 27673 log.cc:824] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:58.112206 27673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb: No bootstrap required, opened a new log
I20260808 05:00:58.114037 27673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2873b39d66874f72aed190c6de4cdbeb" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46829 } }
I20260808 05:00:58.114173 27673 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:58.114205 27673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2873b39d66874f72aed190c6de4cdbeb, State: Initialized, Role: FOLLOWER
I20260808 05:00:58.114327 27673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2873b39d66874f72aed190c6de4cdbeb" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46829 } }
I20260808 05:00:58.114404 27673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:00:58.114436 27673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:00:58.114485 27673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:58.115106 27673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2873b39d66874f72aed190c6de4cdbeb" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46829 } }
I20260808 05:00:58.115209 27673 leader_election.cc:302] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2873b39d66874f72aed190c6de4cdbeb; no voters:
I20260808 05:00:58.115377 27673 leader_election.cc:288] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:00:58.115453 27682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:00:58.115571 27682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [term 1 LEADER]: Becoming Leader. State: Replica: 2873b39d66874f72aed190c6de4cdbeb, State: Running, Role: LEADER
I20260808 05:00:58.115671 27673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:00:58.115715 27682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2873b39d66874f72aed190c6de4cdbeb" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46829 } }
Aug 08 05:00:58 dist-test-slave-gjw7 krb5kdc[27525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:58.116909 27683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2873b39d66874f72aed190c6de4cdbeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2873b39d66874f72aed190c6de4cdbeb" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46829 } } }
I20260808 05:00:58.116997 27683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:58.117369 27684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2873b39d66874f72aed190c6de4cdbeb. Latest consensus state: current_term: 1 leader_uuid: "2873b39d66874f72aed190c6de4cdbeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2873b39d66874f72aed190c6de4cdbeb" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46829 } } }
I20260808 05:00:58.117443 27684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb [sys.catalog]: This master's current role is: LEADER
W20260808 05:00:58.117476 27710 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:00:58.117532 27710 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:00:58.117594 27711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:00:58.117796 27711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:00:58.119335 27711 catalog_manager.cc:1347] Generated new cluster ID: 89f5a65974d44b68aaf246ceca490e5f
I20260808 05:00:58.119386 27711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:00:58.121450 27677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:58.110717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55540 (local address 127.3.174.190:46829)
0808 05:00:58.110895 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:58.110903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:58.110927 (+ 24us) server_negotiation.cc:407] Connection header received
0808 05:00:58.111360 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:58.111370 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:58.111428 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:58.111550 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:58.112151 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:58.113073 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:58.113854 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:58.114276 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:58.114369 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:58.117040 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:58.117044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:58.117045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:58.117057 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:58.119609 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:58.119963 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:58.119966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:58.119967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:58.120060 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:58.120230 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:58.120231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:58.120231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:58.120499 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:58.120625 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:58.121079 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 05:00:58.121217 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260808 05:00:58.122556 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:00:58.128561 27711 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:00:58.129046 27711 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:00:58.134721 27711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2873b39d66874f72aed190c6de4cdbeb: Generated new TSK 0
I20260808 05:00:58.134874 27711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:00:58.176321 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 27548
2026-08-08T05:00:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1501 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:00:58 dist-test-slave-gjw7 krb5kdc[27723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:00:58 dist-test-slave-gjw7 krb5kdc[27723](info): set up 2 sockets
Aug 08 05:00:58 dist-test-slave-gjw7 krb5kdc[27723](info): commencing operation
krb5kdc: starting...
W20260808 05:00:59.374859 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:00:59 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:00:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:00:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:00:59.499791 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38397
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:38397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:59.587863 27747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:59.588080 27747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:59.588110 27747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:59.589273 27747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:00:59.589354 27747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:59.589372 27747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:59.589388 27747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:00:59.589409 27747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:00:59.590899 27747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:38397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:59.591171 27747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:59.591382 27747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:59.596134 27755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:59.596134 27757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:59.596297 27747 server_base.cc:1034] running on GCE node
W20260808 05:00:59.596156 27754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:59.596685 27747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:59.596969 27747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:59.598091 27747 hybrid_clock.cc:648] HybridClock initialized: now 1786165259598055 us; error 56 us; skew 500 ppm
Aug 08 05:00:59 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165259, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:59.600445 27747 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:00:59.600878 27747 webserver.cc:460] Webserver started at http://127.3.174.190:43373/ using document root <none> and password file <none>
I20260808 05:00:59.601078 27747 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:59.601126 27747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:59.601250 27747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:00:59.602335 27747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "c8382fe5abf14d84b548d57ce2782f47"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "f03fc049ce996a218aded5b33df96428"
server_key_iv: "68f3d06f42bb4dfd2751dd3e72f67df7"
server_key_version: "encryptionkey@0"
I20260808 05:00:59.602671 27747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "c8382fe5abf14d84b548d57ce2782f47"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "f03fc049ce996a218aded5b33df96428"
server_key_iv: "68f3d06f42bb4dfd2751dd3e72f67df7"
server_key_version: "encryptionkey@0"
I20260808 05:00:59.603965 27747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 05:00:59.604707 27770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:59.604856 27747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:00:59.604947 27747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "c8382fe5abf14d84b548d57ce2782f47"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "f03fc049ce996a218aded5b33df96428"
server_key_iv: "68f3d06f42bb4dfd2751dd3e72f67df7"
server_key_version: "encryptionkey@0"
I20260808 05:00:59.605011 27747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:59.630225 27747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:59.632226 27747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:59.632373 27747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:59.636509 27747 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:38397
I20260808 05:00:59.636550 27863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:38397 every 8 connection(s)
I20260808 05:00:59.636902 27747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:00:59.637097 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 27747
I20260808 05:00:59.637197 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:00:59.637439 3770 external_mini_cluster.cc:1428] Setting key da15ea63e4b3400ba0f4ff9917d34e02
I20260808 05:00:59.638154 27865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:00:59.641376 27865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Bootstrap starting.
I20260808 05:00:59.642038 27865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Neither blocks nor log segments found. Creating new log.
I20260808 05:00:59.642359 27865 log.cc:824] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Log is configured to *not* fsync() on all Append() calls
I20260808 05:00:59.643152 27865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: No bootstrap required, opened a new log
Aug 08 05:00:59 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:59.645367 27865 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:00:59.645532 27865 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:00:59.645571 27865 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8382fe5abf14d84b548d57ce2782f47, State: Initialized, Role: FOLLOWER
I20260808 05:00:59.645710 27865 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:00:59.645797 27865 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:00:59.645836 27865 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:00:59.645886 27865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:00:59.646554 27865 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:00:59.646663 27865 leader_election.cc:302] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8382fe5abf14d84b548d57ce2782f47; no voters:
I20260808 05:00:59.646826 27865 leader_election.cc:288] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:00:59.646878 27880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:00:59.647047 27880 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 1 LEADER]: Becoming Leader. State: Replica: c8382fe5abf14d84b548d57ce2782f47, State: Running, Role: LEADER
I20260808 05:00:59.647095 27865 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:00:59.647274 27880 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:00:59.647612 27881 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8382fe5abf14d84b548d57ce2782f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } } }
I20260808 05:00:59.647691 27881 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:59.647866 27883 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8382fe5abf14d84b548d57ce2782f47. Latest consensus state: current_term: 1 leader_uuid: "c8382fe5abf14d84b548d57ce2782f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } } }
I20260808 05:00:59.647919 27883 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: This master's current role is: LEADER
I20260808 05:00:59.648499 27891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:00:59.649068 27891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:00:59.649096 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:59.638017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.3.174.190:38397)
0808 05:00:59.638239 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:59.638251 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:59.638284 (+ 33us) server_negotiation.cc:407] Connection header received
0808 05:00:59.638845 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:59.638855 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:59.638924 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:59.639077 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:59.639644 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:59.641034 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:59.641830 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:59.642419 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:59.642529 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:59.645012 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:59.645018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:59.645018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:59.645033 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:59.646994 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:59.647410 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:59.647413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:59.647414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:59.647518 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:59.647749 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:59.647750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:59.647750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:59.648089 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:59.648753 (+ 664us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:59.648840 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 05:00:59.648951 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":131,"threads_started":1}
I20260808 05:00:59.650599 27891 catalog_manager.cc:1347] Generated new cluster ID: 7475274bf4f141bdab7302cb2eab0f92
I20260808 05:00:59.650646 27891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:00:59.659952 27891 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:00:59.660432 27891 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:00:59.665928 27891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Generated new TSK 0
I20260808 05:00:59.666118 27891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 05:00:59.694373 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:38397
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:59.780584 27912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:59.780807 27912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:59.780840 27912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:59.782027 27912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:59.782069 27912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:59.782105 27912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:00:59.783591 27912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:38397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:59.783849 27912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:59.784080 27912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:59.789032 27922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:59.789088 27923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:59.789157 27912 server_base.cc:1034] running on GCE node
W20260808 05:00:59.789050 27926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:59.789448 27912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:59.789697 27912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:59.790836 27912 hybrid_clock.cc:648] HybridClock initialized: now 1786165259790826 us; error 28 us; skew 500 ppm
Aug 08 05:00:59 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165259, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:59.793411 27912 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:00:59.793880 27912 webserver.cc:460] Webserver started at http://127.3.174.129:32863/ using document root <none> and password file <none>
I20260808 05:00:59.794088 27912 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:59.794129 27912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:59.794222 27912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:00:59.795257 27912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "86a4c7a5f81b4e5e846d9155bb3d247a"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "37a2007e55725bc24fbd47c7fad27133"
server_key_iv: "d469d3d7d670a62605d59b61be4b4eaf"
server_key_version: "encryptionkey@0"
I20260808 05:00:59.795567 27912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "86a4c7a5f81b4e5e846d9155bb3d247a"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "37a2007e55725bc24fbd47c7fad27133"
server_key_iv: "d469d3d7d670a62605d59b61be4b4eaf"
server_key_version: "encryptionkey@0"
I20260808 05:00:59.796902 27912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:00:59.797603 27936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:59.797762 27912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:00:59.797853 27912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "86a4c7a5f81b4e5e846d9155bb3d247a"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "37a2007e55725bc24fbd47c7fad27133"
server_key_iv: "d469d3d7d670a62605d59b61be4b4eaf"
server_key_version: "encryptionkey@0"
I20260808 05:00:59.797924 27912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:59.825474 27912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:59.827489 27912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:59.827612 27912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:59.827864 27912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:59.828177 27912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:59.828212 27912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:59.828233 27912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:59.828246 27912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:59.833842 27912 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:33889
I20260808 05:00:59.833914 28145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:33889 every 8 connection(s)
I20260808 05:00:59.834205 27912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 05:00:59 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165259, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:00:59.840988 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 27912
I20260808 05:00:59.841106 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:00:59.841389 3770 external_mini_cluster.cc:1428] Setting key 1d882a547f5871e865976dedd0f85b19
I20260808 05:00:59.842898 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:59.834922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:60139 (local address 127.3.174.190:38397)
0808 05:00:59.834985 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:59.834988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:59.835158 (+ 170us) server_negotiation.cc:407] Connection header received
0808 05:00:59.835643 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:59.835644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:59.835654 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:59.835707 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:59.836626 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:59.837133 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:59.838090 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:59.838279 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:00:59.838334 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:59.840225 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:00:59.840226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:00:59.840226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:00:59.840233 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:00:59.841535 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:59.841997 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:59.841999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:59.841999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:59.842066 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:00:59.842296 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:00:59.842297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:00:59.842297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:00:59.842431 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:00:59.842501 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 05:00:59.842710 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 05:00:59.842802 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 05:00:59.842972 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:59.834880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:60139)
0808 05:00:59.835038 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:59.835081 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:59.835567 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:59.835766 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:59.835769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:59.836358 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:59.836553 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:59.836558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:59.837202 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:59.837202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:59.838022 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:59.838025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:59.838337 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:59.838337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:00:59.838432 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:00:59.838842 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:00:59.838850 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:00:59.840141 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:00:59.841613 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:59.841622 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:59.841624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:59.841929 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:59.842125 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:00:59.842126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:00:59.842126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:00:59.842241 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:00:59.842490 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:00:59.842493 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:00:59.842602 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 05:00:59.842676 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 05:00:59.842796 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260808 05:00:59.843700 28147 heartbeater.cc:347] Connected to a master server at 127.3.174.190:38397
I20260808 05:00:59.843824 28147 heartbeater.cc:464] Registering TS with master...
I20260808 05:00:59.844025 28147 heartbeater.cc:511] Master 127.3.174.190:38397 requested a full tablet report, sending...
I20260808 05:00:59.844534 27794 ts_manager.cc:194] Registered new tserver with Master: 86a4c7a5f81b4e5e846d9155bb3d247a (127.3.174.129:33889)
I20260808 05:00:59.845122 27794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:60139
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 05:00:59.869073 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:38397
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:00:59.962945 28154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:00:59.963161 28154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:00:59.963184 28154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:00:59.964322 28154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:00:59.964355 28154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:00:59.964380 28154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:00:59.965922 28154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:38397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:00:59.966136 28154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:00:59.966341 28154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:00:59.971251 28161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:00:59.971318 28160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:59.971385 28154 server_base.cc:1034] running on GCE node
W20260808 05:00:59.971231 28164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:00:59.971686 28154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:00:59.971920 28154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:00:59.973074 28154 hybrid_clock.cc:648] HybridClock initialized: now 1786165259973061 us; error 30 us; skew 500 ppm
Aug 08 05:00:59 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165259, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:00:59.975561 28154 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:00:59.976022 28154 webserver.cc:460] Webserver started at http://127.3.174.130:43773/ using document root <none> and password file <none>
I20260808 05:00:59.976224 28154 fs_manager.cc:359] Metadata directory not provided
I20260808 05:00:59.976265 28154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:00:59.976372 28154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:00:59.977424 28154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "4a82bd4dbe8545a9985fd1563c118ad9"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "b9d364445cd91484d9b8badc1ed4ce4b"
server_key_iv: "0ca61c2f0951929fe772c32ad2329c16"
server_key_version: "encryptionkey@0"
I20260808 05:00:59.977725 28154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "4a82bd4dbe8545a9985fd1563c118ad9"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "b9d364445cd91484d9b8badc1ed4ce4b"
server_key_iv: "0ca61c2f0951929fe772c32ad2329c16"
server_key_version: "encryptionkey@0"
I20260808 05:00:59.978973 28154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:00:59.979688 28178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:59.979841 28154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:00:59.979924 28154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "4a82bd4dbe8545a9985fd1563c118ad9"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "b9d364445cd91484d9b8badc1ed4ce4b"
server_key_iv: "0ca61c2f0951929fe772c32ad2329c16"
server_key_version: "encryptionkey@0"
I20260808 05:00:59.979996 28154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:00:59.987478 28154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:00:59.989456 28154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:00:59.989580 28154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:00:59.989781 28154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:00:59.990083 28154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:00:59.990116 28154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:59.990144 28154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:00:59.990170 28154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:00:59.996070 28154 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:38775
I20260808 05:00:59.996129 28381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:38775 every 8 connection(s)
I20260808 05:00:59.996474 28154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165259, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:00.005832 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 28154
I20260808 05:01:00.005983 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:01:00.006502 3770 external_mini_cluster.cc:1428] Setting key 93f94e6e76f33eaef39290f634fee461
I20260808 05:01:00.007680 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:59.997158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:57253 (local address 127.3.174.190:38397)
0808 05:00:59.997204 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 05:00:59.997206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:00:59.997450 (+ 244us) server_negotiation.cc:407] Connection header received
0808 05:00:59.997990 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:00:59.997991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:00:59.998004 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:00:59.998062 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:00:59.999039 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:00:59.999548 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.000624 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.000833 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.000897 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.003890 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:00.003892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:00.003892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:00.003900 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:00.005447 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.006259 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.006260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.006261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.006343 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.006789 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.006791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.006791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.006968 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:00.007039 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.007439 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.007565 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 05:01:00.007843 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:00:59.997141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:57253)
0808 05:00:59.997316 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 05:00:59.997365 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 05:00:59.997889 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:00:59.998146 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:00:59.998150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:00:59.998737 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:00:59.998939 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:00:59.998944 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:00:59.999635 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:00:59.999636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.000516 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.000520 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.000935 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.000936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.001064 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.001737 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:00.001747 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:00.003705 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:00.005588 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.005605 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.005608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.006066 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.006437 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.006439 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.006439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.006691 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.007045 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:00.007056 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:00.007246 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.007419 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.007622 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260808 05:01:00.008387 28383 heartbeater.cc:347] Connected to a master server at 127.3.174.190:38397
I20260808 05:01:00.008535 28383 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:00.008777 28383 heartbeater.cc:511] Master 127.3.174.190:38397 requested a full tablet report, sending...
I20260808 05:01:00.009282 27794 ts_manager.cc:194] Registered new tserver with Master: 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775)
I20260808 05:01:00.009727 27794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:57253
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 05:01:00.038479 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:38397
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:00.126411 28390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:00.126619 28390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:00.126641 28390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:00.127787 28390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:00.127828 28390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:00.127852 28390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:01:00.129460 28390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:38397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:00.129698 28390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:00.129907 28390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:00.134938 28405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:00.134991 28399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:00.135008 28390 server_base.cc:1034] running on GCE node
W20260808 05:01:00.134943 28407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:00.135435 28390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:00.135646 28390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:00.136899 28390 hybrid_clock.cc:648] HybridClock initialized: now 1786165260136866 us; error 49 us; skew 500 ppm
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:00.139394 28390 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:01:00.139851 28390 webserver.cc:460] Webserver started at http://127.3.174.131:35795/ using document root <none> and password file <none>
I20260808 05:01:00.140062 28390 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:00.140111 28390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:00.140242 28390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:01:00.141328 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "b622d8824a2e486b945735e7888edf48"
format_stamp: "Formatted at 2026-08-08 05:01:00 on dist-test-slave-gjw7"
server_key: "51ab7717fefbe38ded48dab6c7cafedf"
server_key_iv: "b3a7e3a002c83705cacba259734eb371"
server_key_version: "encryptionkey@0"
I20260808 05:01:00.141688 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "b622d8824a2e486b945735e7888edf48"
format_stamp: "Formatted at 2026-08-08 05:01:00 on dist-test-slave-gjw7"
server_key: "51ab7717fefbe38ded48dab6c7cafedf"
server_key_iv: "b3a7e3a002c83705cacba259734eb371"
server_key_version: "encryptionkey@0"
I20260808 05:01:00.142968 28390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:00.143702 28415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.143872 28390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.143981 28390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "b622d8824a2e486b945735e7888edf48"
format_stamp: "Formatted at 2026-08-08 05:01:00 on dist-test-slave-gjw7"
server_key: "51ab7717fefbe38ded48dab6c7cafedf"
server_key_iv: "b3a7e3a002c83705cacba259734eb371"
server_key_version: "encryptionkey@0"
I20260808 05:01:00.144057 28390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:00.180341 28390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:00.182559 28390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:00.182701 28390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:00.182934 28390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:00.183254 28390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:00.183288 28390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.183319 28390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:00.183341 28390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.189116 28390 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:42595
I20260808 05:01:00.189173 28616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:42595 every 8 connection(s)
I20260808 05:01:00.189520 28390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:01:00.194685 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 28390
I20260808 05:01:00.194768 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 05:01:00.194927 3770 external_mini_cluster.cc:1428] Setting key 7b815d3dd4d1c9a7c762f09cede0d4f5
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:00.198766 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.190342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:41129 (local address 127.3.174.190:38397)
0808 05:01:00.190412 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.190415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.190580 (+ 165us) server_negotiation.cc:407] Connection header received
0808 05:01:00.191122 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.191123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.191136 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.191183 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:00.192160 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.192656 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.193667 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.193859 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.193923 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.196125 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:00.196126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:00.196126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:00.196134 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:00.197536 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.197902 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.197903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.197903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.197960 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.198147 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.198147 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.198147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.198251 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:00.198329 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.198615 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.198696 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 05:01:00.198900 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.190264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:41129)
0808 05:01:00.190449 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.190497 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.191018 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.191270 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.191274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.191860 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:00.192059 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.192064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.192740 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.192741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.193587 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.193590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.193937 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.193938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.194057 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.194520 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:00.194529 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:00.196039 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:00.197615 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.197622 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.197624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.197838 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.198012 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.198012 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.198012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.198105 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.198326 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:00.198330 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:00.198474 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.198563 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.198703 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260808 05:01:00.199373 28618 heartbeater.cc:347] Connected to a master server at 127.3.174.190:38397
I20260808 05:01:00.199465 28618 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:00.199620 28618 heartbeater.cc:511] Master 127.3.174.190:38397 requested a full tablet report, sending...
I20260808 05:01:00.200004 27794 ts_manager.cc:194] Registered new tserver with Master: b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595)
I20260808 05:01:00.200343 27794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:41129
I20260808 05:01:00.206532 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:01:00.213249 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.207929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.3.174.190:38397)
0808 05:01:00.207997 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.208000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.208062 (+ 62us) server_negotiation.cc:407] Connection header received
0808 05:01:00.208108 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.208108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.208117 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.208161 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:00.208699 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.209167 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.209940 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.210125 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.210182 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.211320 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:00.211322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:00.211322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:00.211329 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:00.212394 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.212673 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.212674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.212674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.212718 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.212842 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.212842 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.212842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.212969 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:00.213043 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.213119 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.213176 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 05:01:00.214785 27794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:01:00.216471 27794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:01:00.222287 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.218712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33889 (local address 127.0.0.1:55586)
0808 05:01:00.218874 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.218912 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.218977 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.219283 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.219286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.219288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:00.219339 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.219340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.220296 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.220297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.221212 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.221214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.221991 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.221992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.222075 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.222089 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.222153 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.222208 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":57,"threads_started":1}
I20260808 05:01:00.222530 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.218899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55586 (local address 127.3.174.129:33889)
0808 05:01:00.219106 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.219111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.219121 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:01:00.219160 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.219162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.219178 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.219264 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:00.219390 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.220202 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.221314 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.221915 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.221981 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.222043 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.222406 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.222451 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":93,"threads_started":1}
I20260808 05:01:00.223100 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.218719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:38775 (local address 127.0.0.1:35354)
0808 05:01:00.219188 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.219198 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.219249 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.219587 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.219588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.219590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:00.219638 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.219640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.220978 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.220979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.222083 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.222086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.222855 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.222856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.222935 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.222941 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.222989 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.223032 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":88,"threads_started":1}
I20260808 05:01:00.223125 28645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.218970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.3.174.130:38775)
0808 05:01:00.219141 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.219145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.219425 (+ 280us) server_negotiation.cc:407] Connection header received
0808 05:01:00.219473 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.219476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.219494 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.219718 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:00.219788 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.220876 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.222164 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.222777 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.222842 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.222895 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.223014 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.223051 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260808 05:01:00.223266 28030 tablet_service.cc:1467] Processing CreateTablet for tablet 7728218c7684456dadee61fba60704bb (DEFAULT_TABLE table=test-table [id=3ee3733259f74e0a80034a9efb86afce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:00.223641 28030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7728218c7684456dadee61fba60704bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:00.223806 28259 tablet_service.cc:1467] Processing CreateTablet for tablet 7728218c7684456dadee61fba60704bb (DEFAULT_TABLE table=test-table [id=3ee3733259f74e0a80034a9efb86afce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:00.224066 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.218712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:42595 (local address 127.0.0.1:60238)
0808 05:01:00.219142 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.219152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.219201 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.219618 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.219619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.219622 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:00.219678 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.219680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.221344 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.221345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.222340 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.222342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.223300 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.223301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.223906 (+ 605us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.223911 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.223961 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.224000 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260808 05:01:00.224144 28259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7728218c7684456dadee61fba60704bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:00.224289 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.218967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.3.174.131:42595)
0808 05:01:00.219399 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.219403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.219412 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:01:00.219449 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.219450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.219465 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.219543 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:00.219746 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.220861 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.222515 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.223212 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.223280 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.223345 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.224167 (+ 822us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.224214 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":94,"threads_started":1}
I20260808 05:01:00.226478 28647 tablet_bootstrap.cc:492] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Bootstrap starting.
I20260808 05:01:00.226765 28650 tablet_bootstrap.cc:492] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: Bootstrap starting.
I20260808 05:01:00.227057 28506 tablet_service.cc:1467] Processing CreateTablet for tablet 7728218c7684456dadee61fba60704bb (DEFAULT_TABLE table=test-table [id=3ee3733259f74e0a80034a9efb86afce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:00.227324 28647 tablet_bootstrap.cc:654] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:00.227350 28506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7728218c7684456dadee61fba60704bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:00.227370 28650 tablet_bootstrap.cc:654] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:00.227906 28647 log.cc:824] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:00.227906 28650 log.cc:824] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:00.228642 28647 tablet_bootstrap.cc:492] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: No bootstrap required, opened a new log
I20260808 05:01:00.228699 28647 ts_tablet_manager.cc:1397] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:01:00.229094 28650 tablet_bootstrap.cc:492] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: No bootstrap required, opened a new log
I20260808 05:01:00.229164 28650 ts_tablet_manager.cc:1397] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:01:00.230021 28654 tablet_bootstrap.cc:492] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: Bootstrap starting.
I20260808 05:01:00.230610 28654 tablet_bootstrap.cc:654] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:00.230626 28647 raft_consensus.cc:348] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.230823 28647 raft_consensus.cc:374] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:00.230895 28647 raft_consensus.cc:729] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a4c7a5f81b4e5e846d9155bb3d247a, State: Initialized, Role: FOLLOWER
I20260808 05:01:00.231021 28647 consensus_queue.cc:260] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.231164 28654 log.cc:824] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:00.231108 28650 raft_consensus.cc:348] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.231276 28650 raft_consensus.cc:374] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:00.231315 28647 ts_tablet_manager.cc:1428] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 05:01:00.231333 28650 raft_consensus.cc:729] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a82bd4dbe8545a9985fd1563c118ad9, State: Initialized, Role: FOLLOWER
I20260808 05:01:00.231441 28147 heartbeater.cc:503] Master 127.3.174.190:38397 was elected leader, sending a full tablet report...
I20260808 05:01:00.231741 28650 consensus_queue.cc:260] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.231931 28654 tablet_bootstrap.cc:492] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: No bootstrap required, opened a new log
I20260808 05:01:00.232039 28654 ts_tablet_manager.cc:1397] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:01:00.232183 28383 heartbeater.cc:503] Master 127.3.174.190:38397 was elected leader, sending a full tablet report...
I20260808 05:01:00.232172 28650 ts_tablet_manager.cc:1428] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 05:01:00.233723 28654 raft_consensus.cc:348] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.233853 28654 raft_consensus.cc:374] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:00.233873 28654 raft_consensus.cc:729] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b622d8824a2e486b945735e7888edf48, State: Initialized, Role: FOLLOWER
I20260808 05:01:00.233991 28654 consensus_queue.cc:260] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.234247 28654 ts_tablet_manager.cc:1428] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 05:01:00.234293 28618 heartbeater.cc:503] Master 127.3.174.190:38397 was elected leader, sending a full tablet report...
W20260808 05:01:00.247051 28384 tablet.cc:2369] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 05:01:00.334930 28148 tablet.cc:2369] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:01:00.343284 28658 raft_consensus.cc:482] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:00.343431 28658 raft_consensus.cc:504] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.343819 28658 leader_election.cc:288] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86a4c7a5f81b4e5e846d9155bb3d247a (127.3.174.129:33889), b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595)
I20260808 05:01:00.346796 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.344072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:45865 (local address 127.3.174.131:42595)
0808 05:01:00.344130 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.344132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.344143 (+ 11us) server_negotiation.cc:407] Connection header received
0808 05:01:00.344169 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.344170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.344179 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.344227 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:00.344413 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.344986 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.346014 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.346455 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.346509 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.346516 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.346673 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.346735 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 05:01:00.346787 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.344000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:42595 (local address 127.3.174.130:45865)
0808 05:01:00.344052 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.344065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.344110 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.344254 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.344256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.344258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:00.344335 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.344337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.345060 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.345061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.345920 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.345923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.346522 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.346523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.346609 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.346614 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.346664 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.346708 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 05:01:00.346931 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.344088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:32953 (local address 127.3.174.129:33889)
0808 05:01:00.344146 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.344149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.344418 (+ 269us) server_negotiation.cc:407] Connection header received
0808 05:01:00.344475 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.344476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.344484 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.344520 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:00.344653 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.345148 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.346095 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.346531 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.346600 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.346610 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.346804 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.346848 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 05:01:00.346951 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.344004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33889 (local address 127.3.174.130:32953)
0808 05:01:00.344360 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.344369 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.344426 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.344533 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.344534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.344536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:00.344599 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.344600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.345222 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.345223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.345989 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.345992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.346638 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.346639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.346712 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.346715 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.346790 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.346862 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":152,"thread_start_us":135,"threads_started":1}
I20260808 05:01:00.347201 28543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7728218c7684456dadee61fba60704bb" candidate_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b622d8824a2e486b945735e7888edf48" is_pre_election: true
I20260808 05:01:00.347368 28543 raft_consensus.cc:2454] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a82bd4dbe8545a9985fd1563c118ad9 in term 0.
I20260808 05:01:00.347335 28065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7728218c7684456dadee61fba60704bb" candidate_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" is_pre_election: true
I20260808 05:01:00.347512 28065 raft_consensus.cc:2454] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a82bd4dbe8545a9985fd1563c118ad9 in term 0.
I20260808 05:01:00.347618 28183 leader_election.cc:302] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a82bd4dbe8545a9985fd1563c118ad9, b622d8824a2e486b945735e7888edf48; no voters:
I20260808 05:01:00.347739 28658 raft_consensus.cc:2781] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:01:00.347796 28658 raft_consensus.cc:482] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:01:00.347820 28658 raft_consensus.cc:3037] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:00.348608 28658 raft_consensus.cc:504] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.348721 28658 leader_election.cc:288] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [CANDIDATE]: Term 1 election: Requested vote from peers 86a4c7a5f81b4e5e846d9155bb3d247a (127.3.174.129:33889), b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595)
I20260808 05:01:00.348884 28065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7728218c7684456dadee61fba60704bb" candidate_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a"
I20260808 05:01:00.348927 28543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7728218c7684456dadee61fba60704bb" candidate_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b622d8824a2e486b945735e7888edf48"
I20260808 05:01:00.348970 28065 raft_consensus.cc:3037] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:00.348992 28543 raft_consensus.cc:3037] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:00.349861 28065 raft_consensus.cc:2454] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a82bd4dbe8545a9985fd1563c118ad9 in term 1.
I20260808 05:01:00.349861 28543 raft_consensus.cc:2454] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a82bd4dbe8545a9985fd1563c118ad9 in term 1.
I20260808 05:01:00.350081 28185 leader_election.cc:302] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a82bd4dbe8545a9985fd1563c118ad9, 86a4c7a5f81b4e5e846d9155bb3d247a; no voters:
I20260808 05:01:00.350260 28658 raft_consensus.cc:2781] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:01:00.350440 28658 raft_consensus.cc:686] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 1 LEADER]: Becoming Leader. State: Replica: 4a82bd4dbe8545a9985fd1563c118ad9, State: Running, Role: LEADER
I20260808 05:01:00.350533 28658 consensus_queue.cc:237] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.351377 27794 catalog_manager.cc:5495] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130). New cstate: current_term: 1 leader_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } health_report { overall_health: HEALTHY } } }
I20260808 05:01:00.368249 28645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.365987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.3.174.130:38775)
0808 05:01:00.366045 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.366047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.366056 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:01:00.366103 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.366104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.366126 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.366186 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:01:00.366335 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.366795 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.367511 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.367699 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.367753 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.367899 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:01:00.368022 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:01:00.368092 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.368129 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.368151 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 05:01:00.370111 28065 raft_consensus.cc:1261] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Refusing update from remote peer 4a82bd4dbe8545a9985fd1563c118ad9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:01:00.370121 28543 raft_consensus.cc:1261] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [term 1 FOLLOWER]: Refusing update from remote peer 4a82bd4dbe8545a9985fd1563c118ad9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:01:00.370468 28658 consensus_queue.cc:1035] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:01:00.370615 28666 consensus_queue.cc:1035] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:01:00.372984 28669 mvcc.cc:205] Tried to move back new op lower bound from 7316132906473148416 to 7316132906396282880. Current Snapshot: MvccSnapshot[applied={T|T < 7316132906473148416}]
I20260808 05:01:00.373560 28671 mvcc.cc:205] Tried to move back new op lower bound from 7316132906473148416 to 7316132906396282880. Current Snapshot: MvccSnapshot[applied={T|T < 7316132906473148416}]
I20260808 05:01:00.375452 27794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:01:00.375552 27794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:01:00.376296 27794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Sending DeleteTablet for 3 replicas of tablet 7728218c7684456dadee61fba60704bb
I20260808 05:01:00.376608 28506 tablet_service.cc:1514] Processing DeleteTablet for tablet 7728218c7684456dadee61fba60704bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:00 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:60238
I20260808 05:01:00.376626 28030 tablet_service.cc:1514] Processing DeleteTablet for tablet 7728218c7684456dadee61fba60704bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:00 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:55586
I20260808 05:01:00.376673 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 27912
I20260808 05:01:00.376642 28259 tablet_service.cc:1514] Processing DeleteTablet for tablet 7728218c7684456dadee61fba60704bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:00 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:35354
I20260808 05:01:00.389045 28694 tablet_replica.cc:330] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: stopping tablet replica
I20260808 05:01:00.389262 28694 raft_consensus.cc:2229] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:01:00.389488 28694 raft_consensus.cc:2258] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:00.391947 28694 ts_tablet_manager.cc:1905] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:00.393601 28694 ts_tablet_manager.cc:1918] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:00.393803 28694 log.cc:1196] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/7728218c7684456dadee61fba60704bb
I20260808 05:01:00.394260 28694 ts_tablet_manager.cc:1939] T 7728218c7684456dadee61fba60704bb P 4a82bd4dbe8545a9985fd1563c118ad9: Deleting consensus metadata
I20260808 05:01:00.397686 27771 catalog_manager.cc:4841] TS 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775): tablet 7728218c7684456dadee61fba60704bb (table test-table [id=3ee3733259f74e0a80034a9efb86afce]) successfully deleted
I20260808 05:01:00.398118 28693 tablet_replica.cc:330] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: stopping tablet replica
I20260808 05:01:00.398270 28693 raft_consensus.cc:2229] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:01:00.398415 28693 raft_consensus.cc:2258] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:00.398864 28693 ts_tablet_manager.cc:1905] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:00.400144 28693 ts_tablet_manager.cc:1918] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:00.400216 28693 log.cc:1196] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/7728218c7684456dadee61fba60704bb
I20260808 05:01:00.400484 28693 ts_tablet_manager.cc:1939] T 7728218c7684456dadee61fba60704bb P b622d8824a2e486b945735e7888edf48: Deleting consensus metadata
I20260808 05:01:00.400930 27772 catalog_manager.cc:4841] TS b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595): tablet 7728218c7684456dadee61fba60704bb (table test-table [id=3ee3733259f74e0a80034a9efb86afce]) successfully deleted
W20260808 05:01:00.402205 27777 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:33889 (error 108)
I20260808 05:01:00.402765 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.402629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33889 (local address 127.0.0.1:55594)
0808 05:01:00.402686 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.402729 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 05:01:00.402784 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 28154
W20260808 05:01:00.405452 27777 catalog_manager.cc:4584] TS 86a4c7a5f81b4e5e846d9155bb3d247a (127.3.174.129:33889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7728218c7684456dadee61fba60704bb: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33889: connect: Connection refused (error 111)
I20260808 05:01:00.413465 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 28390
I20260808 05:01:00.424206 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 27747
I20260808 05:01:00.439821 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38397
--webserver_interface=127.3.174.190
--webserver_port=43373
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:38397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:00.523798 28697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:00.523994 28697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:00.524024 28697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:00.525153 28697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:01:00.525192 28697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:00.525210 28697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:00.525226 28697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:01:00.525241 28697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:01:00.526749 28697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:38397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=43373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:00.526988 28697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:00.527184 28697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:00.537307 28712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:00.537263 28711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:00.537258 28716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:00.537353 28697 server_base.cc:1034] running on GCE node
I20260808 05:01:00.537709 28697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:00.538017 28697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:00.539192 28697 hybrid_clock.cc:648] HybridClock initialized: now 1786165260539173 us; error 32 us; skew 500 ppm
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:00.541762 28697 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:01:00.542191 28697 webserver.cc:460] Webserver started at http://127.3.174.190:43373/ using document root <none> and password file <none>
I20260808 05:01:00.542412 28697 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:00.542465 28697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:00.543798 28697 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:01:00.544423 28726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.544595 28697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:01:00.544683 28697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "c8382fe5abf14d84b548d57ce2782f47"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "f03fc049ce996a218aded5b33df96428"
server_key_iv: "68f3d06f42bb4dfd2751dd3e72f67df7"
server_key_version: "encryptionkey@0"
I20260808 05:01:00.545105 28697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:00.555734 28697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:00.557829 28697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:00.557938 28697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:00.561889 28697 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:38397
I20260808 05:01:00.561951 28812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:38397 every 8 connection(s)
I20260808 05:01:00.562319 28697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:01:00.564189 28814 sys_catalog.cc:263] Verifying existing consensus state
I20260808 05:01:00.565044 28814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Bootstrap starting.
I20260808 05:01:00.565456 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 28697
I20260808 05:01:00.565723 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:33889
--local_ip_for_outbound_sockets=127.3.174.129
--tserver_master_addrs=127.3.174.190:38397
--webserver_port=32863
--webserver_interface=127.3.174.129
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:01:00.574100 28814 log.cc:824] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:00.576094 28814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 05:01:00.576375 28814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Bootstrap complete.
I20260808 05:01:00.578642 28814 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:01:00.578884 28814 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8382fe5abf14d84b548d57ce2782f47, State: Initialized, Role: FOLLOWER
I20260808 05:01:00.579044 28814 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:01:00.579106 28814 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:01:00.579160 28814 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:01:00.579209 28814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 1 FOLLOWER]: Advancing to term 2
I20260808 05:01:00.580281 28814 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:01:00.580385 28814 leader_election.cc:302] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8382fe5abf14d84b548d57ce2782f47; no voters:
I20260808 05:01:00.580564 28814 leader_election.cc:288] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 05:01:00.580615 28822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 05:01:00.580773 28822 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 2 LEADER]: Becoming Leader. State: Replica: c8382fe5abf14d84b548d57ce2782f47, State: Running, Role: LEADER
I20260808 05:01:00.580856 28814 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:01:00.580902 28822 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:01:00.581183 28822 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8382fe5abf14d84b548d57ce2782f47. Latest consensus state: current_term: 2 leader_uuid: "c8382fe5abf14d84b548d57ce2782f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } } }
I20260808 05:01:00.581470 28822 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:00.581393 28823 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c8382fe5abf14d84b548d57ce2782f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } } }
I20260808 05:01:00.581861 28823 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:00.581943 28839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:01:00.582809 28839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:01:00.582963 28839 catalog_manager.cc:1259] Loaded cluster ID: 7475274bf4f141bdab7302cb2eab0f92
I20260808 05:01:00.583165 28839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:01:00.583425 28839 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 05:01:00.588184 28839 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 05:01:00.590739 28839 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:01:00.590957 28839 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Loaded TSK: 0
I20260808 05:01:00.591185 28839 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 05:01:00.653997 28820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:00.654213 28820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:00.654247 28820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:00.655411 28820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:00.655453 28820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:00.655488 28820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:01:00.657080 28820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:33889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=32863
--tserver_master_addrs=127.3.174.190:38397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:00.657514 28820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:00.657783 28820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:00.662755 28867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:00.662782 28874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:00.662844 28820 server_base.cc:1034] running on GCE node
W20260808 05:01:00.662997 28869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:00.663226 28820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:00.663465 28820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:00.664618 28820 hybrid_clock.cc:648] HybridClock initialized: now 1786165260664599 us; error 34 us; skew 500 ppm
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:00.667259 28820 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:01:00.667729 28820 webserver.cc:460] Webserver started at http://127.3.174.129:32863/ using document root <none> and password file <none>
I20260808 05:01:00.667956 28820 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:00.668020 28820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:00.669370 28820 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:00.669996 28885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.670163 28820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:01:00.670274 28820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "86a4c7a5f81b4e5e846d9155bb3d247a"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "37a2007e55725bc24fbd47c7fad27133"
server_key_iv: "d469d3d7d670a62605d59b61be4b4eaf"
server_key_version: "encryptionkey@0"
I20260808 05:01:00.670701 28820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:00.697091 28820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:00.699347 28820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:00.699474 28820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:00.699731 28820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:00.700265 28897 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 05:01:00.701889 28820 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 05:01:00.701941 28820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 05:01:00.701968 28820 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 05:01:00.702616 28820 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 05:01:00.702657 28820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 05:01:00.702729 28897 tablet_bootstrap.cc:492] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Bootstrap starting.
I20260808 05:01:00.710363 28820 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:33889
I20260808 05:01:00.710891 28820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:01:00.712006 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 28820
I20260808 05:01:00.712280 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:38775
--local_ip_for_outbound_sockets=127.3.174.130
--tserver_master_addrs=127.3.174.190:38397
--webserver_port=43773
--webserver_interface=127.3.174.130
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:01:00.713589 29097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:33889 every 8 connection(s)
I20260808 05:01:00.714023 28897 log.cc:824] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:00.716552 28897 tablet_bootstrap.cc:492] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 05:01:00.716845 28897 tablet_bootstrap.cc:492] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Bootstrap complete.
I20260808 05:01:00.717978 28897 ts_tablet_manager.cc:1397] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.005s
I20260808 05:01:00.719404 28897 raft_consensus.cc:348] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.719743 28897 raft_consensus.cc:729] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a4c7a5f81b4e5e846d9155bb3d247a, State: Initialized, Role: FOLLOWER
I20260808 05:01:00.719928 28897 consensus_queue.cc:260] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } } peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } }
I20260808 05:01:00.720220 28897 ts_tablet_manager.cc:1428] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:00.723964 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.711809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:40449)
0808 05:01:00.713853 (+ 2044us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.713903 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.714541 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.715180 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.715184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.715542 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:00.715801 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.715808 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.717005 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.717007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.717876 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.717879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.718546 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.718548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.718689 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.719201 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:00.719211 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:00.720663 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:00.722426 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.722434 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.722436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.722653 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.722889 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.722890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.722891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.722995 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.723348 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:00.723355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:00.723484 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.723572 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.723761 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1991,"thread_start_us":114,"threads_started":1}
I20260808 05:01:00.723965 29107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.713466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:40449 (local address 127.3.174.190:38397)
0808 05:01:00.714484 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.714495 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.714539 (+ 44us) server_negotiation.cc:407] Connection header received
0808 05:01:00.714833 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.714841 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.714899 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.715033 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:00.715954 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.716858 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.717975 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.718441 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.718545 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.720765 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:00.720769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:00.720770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:00.720782 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:00.722347 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.722727 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.722731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.722732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.722830 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.723051 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.723052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.723052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.723284 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:00.723357 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.723636 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.723761 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":962,"thread_start_us":76,"threads_started":1}
I20260808 05:01:00.724795 29098 heartbeater.cc:347] Connected to a master server at 127.3.174.190:38397
I20260808 05:01:00.724905 29098 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:00.725100 29098 heartbeater.cc:511] Master 127.3.174.190:38397 requested a full tablet report, sending...
I20260808 05:01:00.725885 28758 ts_manager.cc:194] Registered new tserver with Master: 86a4c7a5f81b4e5e846d9155bb3d247a (127.3.174.129:33889)
I20260808 05:01:00.726944 28758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:40449
I20260808 05:01:00.727913 29098 heartbeater.cc:503] Master 127.3.174.190:38397 was elected leader, sending a full tablet report...
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.129@KRBTEST.COM
I20260808 05:01:00.735473 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.726476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33889 (local address 127.0.0.1:55606)
0808 05:01:00.726625 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.726661 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.726723 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.726975 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.726979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.727296 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:00.727355 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.727357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.728191 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.728192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.728886 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.728889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.729355 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.729356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.729440 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.729849 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:00.729855 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:00.731097 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:00.734380 (+ 3283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.734387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.734388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.734571 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.734793 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.734794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.734794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.734861 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.735149 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:00.735156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:00.735225 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.735300 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.735402 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260808 05:01:00.735534 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.726682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.3.174.129:33889)
0808 05:01:00.726829 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.726833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.726843 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:01:00.726879 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.726881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.726894 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.726966 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:00.727428 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.728133 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.728961 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.729255 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.729352 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.731177 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:00.731185 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:00.731185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:00.731192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:00.734300 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.734641 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.734643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.734644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.734709 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.734922 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.734922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.734923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.735090 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:00.735163 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.735350 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.735446 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260808 05:01:00.736001 28981 tablet_service.cc:1514] Processing DeleteTablet for tablet 7728218c7684456dadee61fba60704bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:00 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:55606
I20260808 05:01:00.736500 29116 tablet_replica.cc:330] stopping tablet replica
I20260808 05:01:00.736608 29116 raft_consensus.cc:2229] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:01:00.736675 29116 raft_consensus.cc:2258] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:00.736986 29116 ts_tablet_manager.cc:1905] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:00.739379 29116 ts_tablet_manager.cc:1918] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:00.739455 29116 log.cc:1196] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/7728218c7684456dadee61fba60704bb
I20260808 05:01:00.739662 29116 ts_tablet_manager.cc:1939] T 7728218c7684456dadee61fba60704bb P 86a4c7a5f81b4e5e846d9155bb3d247a: Deleting consensus metadata
I20260808 05:01:00.740141 28737 catalog_manager.cc:4841] TS 86a4c7a5f81b4e5e846d9155bb3d247a (127.3.174.129:33889): tablet 7728218c7684456dadee61fba60704bb (table test-table [id=3ee3733259f74e0a80034a9efb86afce]) successfully deleted
W20260808 05:01:00.802356 29108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:00.802559 29108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:00.802580 29108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:00.803714 29108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:00.803752 29108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:00.803777 29108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:01:00.805224 29108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:38775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=43773
--tserver_master_addrs=127.3.174.190:38397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:00.805464 29108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:00.805648 29108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:00.810812 29129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:00.810891 29122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:00.811050 29108 server_base.cc:1034] running on GCE node
W20260808 05:01:00.810815 29123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:00.811362 29108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:00.811609 29108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:00.812777 29108 hybrid_clock.cc:648] HybridClock initialized: now 1786165260812764 us; error 28 us; skew 500 ppm
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:00.815397 29108 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:01:00.815873 29108 webserver.cc:460] Webserver started at http://127.3.174.130:43773/ using document root <none> and password file <none>
I20260808 05:01:00.816112 29108 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:00.816222 29108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:00.817608 29108 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 05:01:00.818225 29143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.818403 29108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:01:00.818491 29108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "4a82bd4dbe8545a9985fd1563c118ad9"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "b9d364445cd91484d9b8badc1ed4ce4b"
server_key_iv: "0ca61c2f0951929fe772c32ad2329c16"
server_key_version: "encryptionkey@0"
I20260808 05:01:00.818840 29108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:00.834434 29108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:00.836404 29108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:00.836537 29108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:00.836747 29108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:00.837042 29108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:00.837077 29108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.837107 29108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:00.837131 29108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.842878 29108 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:38775
I20260808 05:01:00.842933 29352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:38775 every 8 connection(s)
I20260808 05:01:00.843266 29108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:00.850558 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 29108
I20260808 05:01:00.850822 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:42595
--local_ip_for_outbound_sockets=127.3.174.131
--tserver_master_addrs=127.3.174.190:38397
--webserver_port=35795
--webserver_interface=127.3.174.131
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:01:00.852324 29107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.843971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:52599 (local address 127.3.174.190:38397)
0808 05:01:00.844035 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.844037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.844234 (+ 197us) server_negotiation.cc:407] Connection header received
0808 05:01:00.844784 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.844785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.844809 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.844871 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:00.845883 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.846387 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.847333 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.847562 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.847619 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.849714 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:00.849715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:00.849716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:00.849724 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:00.851064 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.851429 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.851430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.851430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.851491 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.851701 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.851702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.851702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.851823 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:00.851878 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.852135 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.852229 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 05:01:00.852483 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.843949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:52599)
0808 05:01:00.844111 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.844148 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.844677 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.844956 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.844961 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.845595 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:00.845784 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.845790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.846469 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.846470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.847249 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.847251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.847632 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.847633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.847780 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.848215 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:00.848223 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:00.849619 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:00.851141 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.851149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.851151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.851365 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.851543 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.851544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.851544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.851647 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.851878 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:00.851885 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:00.852018 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.852106 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.852255 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260808 05:01:00.853070 29355 heartbeater.cc:347] Connected to a master server at 127.3.174.190:38397
I20260808 05:01:00.853178 29355 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:00.853390 29355 heartbeater.cc:511] Master 127.3.174.190:38397 requested a full tablet report, sending...
I20260808 05:01:00.853788 28758 ts_manager.cc:194] Registered new tserver with Master: 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775)
I20260808 05:01:00.854148 28758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:52599
W20260808 05:01:00.934276 29362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:00.934480 29362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:00.934512 29362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:00.935648 29362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:00.935690 29362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:00.935724 29362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:01:00.937227 29362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:42595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=35795
--tserver_master_addrs=127.3.174.190:38397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:00.937491 29362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:00.937711 29362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:00.942507 29370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:00.942561 29362 server_base.cc:1034] running on GCE node
W20260808 05:01:00.942514 29372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:00.942519 29375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:00.942888 29362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:00.943112 29362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:00.944293 29362 hybrid_clock.cc:648] HybridClock initialized: now 1786165260944280 us; error 30 us; skew 500 ppm
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:00.946715 29362 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:01:00.947186 29362 webserver.cc:460] Webserver started at http://127.3.174.131:35795/ using document root <none> and password file <none>
I20260808 05:01:00.947428 29362 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:00.947494 29362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:00.948863 29362 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:01:00.949412 29389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.949558 29362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:01:00.949643 29362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "b622d8824a2e486b945735e7888edf48"
format_stamp: "Formatted at 2026-08-08 05:01:00 on dist-test-slave-gjw7"
server_key: "51ab7717fefbe38ded48dab6c7cafedf"
server_key_iv: "b3a7e3a002c83705cacba259734eb371"
server_key_version: "encryptionkey@0"
I20260808 05:01:00.949977 29362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:00.964480 29362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:00.966440 29362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:00.966578 29362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:00.966805 29362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:00.967109 29362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:00.967144 29362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.967173 29362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:00.967197 29362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:00.972699 29362 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:42595
I20260808 05:01:00.972760 29591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:42595 every 8 connection(s)
I20260808 05:01:00.973075 29362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:01:00.976908 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 29362
Aug 08 05:01:00 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165260, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:00.982296 29107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.973766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:41103 (local address 127.3.174.190:38397)
0808 05:01:00.973831 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.973833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.974032 (+ 199us) server_negotiation.cc:407] Connection header received
0808 05:01:00.974526 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.974527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.974538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.974591 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:00.975487 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.976010 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.976933 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.977172 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.977238 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.979353 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:00.979355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:00.979355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:00.979362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:00.980828 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.981312 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.981314 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.981314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.981379 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.981658 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.981659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.981659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.981761 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:00.981828 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.982155 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.982227 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 05:01:00.982846 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.973763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:41103)
0808 05:01:00.973919 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.973956 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.974435 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.974656 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.974659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.975205 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:00.975397 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.975402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.976080 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.976081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.976859 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.976861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.977258 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.977259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.977449 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.977929 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:00.977940 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:00.979268 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:00.980901 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.980908 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.980910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.981213 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.981443 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:00.981444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:00.981444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:00.981589 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:00.981823 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:00.981830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:00.982012 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.982133 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.982301 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260808 05:01:00.983318 29592 heartbeater.cc:347] Connected to a master server at 127.3.174.190:38397
I20260808 05:01:00.983424 29592 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:00.983624 29592 heartbeater.cc:511] Master 127.3.174.190:38397 requested a full tablet report, sending...
I20260808 05:01:00.983814 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.977246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.3.174.190:38397)
0808 05:01:00.977366 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.977368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.977461 (+ 93us) server_negotiation.cc:407] Connection header received
0808 05:01:00.977520 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.977521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.977529 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.977579 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:00.978211 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.978682 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.979430 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.979664 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.979727 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.980997 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:00.980998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:00.980999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:00.981005 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:00.982614 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.983014 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.983015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.983016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.983071 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:00.983287 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:00.983288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:00.983288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:00.983423 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:00.983600 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.983638 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.983717 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260808 05:01:00.984053 28757 ts_manager.cc:194] Registered new tserver with Master: b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595)
I20260808 05:01:00.984431 28757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:41103
I20260808 05:01:00.984530 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:01:00.987134 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.984856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.3.174.190:38397)
0808 05:01:00.984890 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.984892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.985004 (+ 112us) server_negotiation.cc:407] Connection header received
0808 05:01:00.985073 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.985073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.985111 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.985163 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:01:00.985256 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.985728 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.986437 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.986656 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.986707 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.986856 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:01:00.986948 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:01:00.987012 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.987049 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.987071 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 05:01:00.987419 28757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:01:00.989015 28757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:01:00.991257 28981 tablet_service.cc:1467] Processing CreateTablet for tablet 4736c170a5da4547993f4989b880f4d1 (DEFAULT_TABLE table=test-table [id=648ee178711d4606835d393c8c841232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:00.991477 28981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4736c170a5da4547993f4989b880f4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:00.993062 28897 tablet_bootstrap.cc:492] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Bootstrap starting.
I20260808 05:01:00.993384 28897 tablet_bootstrap.cc:654] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:00.994128 28897 tablet_bootstrap.cc:492] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: No bootstrap required, opened a new log
I20260808 05:01:00.994177 28897 ts_tablet_manager.cc:1397] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 05:01:00.994346 28897 raft_consensus.cc:348] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:00.994426 28897 raft_consensus.cc:374] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:00.994473 28897 raft_consensus.cc:729] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a4c7a5f81b4e5e846d9155bb3d247a, State: Initialized, Role: FOLLOWER
I20260808 05:01:00.994519 28897 consensus_queue.cc:260] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:00.994618 28897 ts_tablet_manager.cc:1428] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 05:01:00.994820 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.990861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:42595 (local address 127.0.0.1:60242)
0808 05:01:00.990914 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.990923 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.990954 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.991442 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.991443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.991445 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:00.991489 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.991490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.992673 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.992673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.993726 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.993727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.994557 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.994558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.994633 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.994658 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.994689 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.994738 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 05:01:00.994935 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.991119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.3.174.130:38775)
0808 05:01:00.991269 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.991273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.991282 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:01:00.991318 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.991319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.991332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.991575 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:00.991647 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.992378 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.993719 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.994531 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.994602 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.994667 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.994805 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.994847 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260808 05:01:00.995129 29602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.990900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:38775 (local address 127.0.0.1:35380)
0808 05:01:00.991050 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:00.991060 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:00.991111 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:00.991449 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:00.991450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:00.991452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:00.991500 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.991501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.992468 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.992469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.993614 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:00.993616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.994652 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.994653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:00.994727 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.994730 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:01:00.994781 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 05:01:00.994815 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260808 05:01:00.995139 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:00.990929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60242 (local address 127.3.174.131:42595)
0808 05:01:00.991256 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:00.991260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:00.991271 (+ 11us) server_negotiation.cc:407] Connection header received
0808 05:01:00.991308 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:00.991310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:00.991330 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:00.991563 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:00.991630 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.992573 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.993815 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:00.994464 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:00.994926 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:00.994994 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:00.995022 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 05:01:00.995061 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":85,"threads_started":1}
I20260808 05:01:00.995738 29470 tablet_service.cc:1467] Processing CreateTablet for tablet 4736c170a5da4547993f4989b880f4d1 (DEFAULT_TABLE table=test-table [id=648ee178711d4606835d393c8c841232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:00.995760 29234 tablet_service.cc:1467] Processing CreateTablet for tablet 4736c170a5da4547993f4989b880f4d1 (DEFAULT_TABLE table=test-table [id=648ee178711d4606835d393c8c841232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:00.996104 29470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4736c170a5da4547993f4989b880f4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:00.996114 29234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4736c170a5da4547993f4989b880f4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:00.998857 29607 tablet_bootstrap.cc:492] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: Bootstrap starting.
I20260808 05:01:00.998862 29609 tablet_bootstrap.cc:492] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: Bootstrap starting.
I20260808 05:01:00.999420 29607 tablet_bootstrap.cc:654] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:00.999441 29609 tablet_bootstrap.cc:654] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:00.999847 29607 log.cc:824] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:00.999871 29609 log.cc:824] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:01.000506 29609 tablet_bootstrap.cc:492] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: No bootstrap required, opened a new log
I20260808 05:01:01.000509 29607 tablet_bootstrap.cc:492] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: No bootstrap required, opened a new log
I20260808 05:01:01.000561 29609 ts_tablet_manager.cc:1397] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:01:01.000566 29607 ts_tablet_manager.cc:1397] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:01:01.001989 29607 raft_consensus.cc:348] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:01.002020 29609 raft_consensus.cc:348] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:01.002137 29609 raft_consensus.cc:374] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:01.002127 29607 raft_consensus.cc:374] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:01.002158 29609 raft_consensus.cc:729] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b622d8824a2e486b945735e7888edf48, State: Initialized, Role: FOLLOWER
I20260808 05:01:01.002161 29607 raft_consensus.cc:729] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a82bd4dbe8545a9985fd1563c118ad9, State: Initialized, Role: FOLLOWER
I20260808 05:01:01.002279 29607 consensus_queue.cc:260] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:01.002279 29609 consensus_queue.cc:260] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:01.002550 29609 ts_tablet_manager.cc:1428] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:01:01.002557 29607 ts_tablet_manager.cc:1428] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:01:01.002588 29355 heartbeater.cc:503] Master 127.3.174.190:38397 was elected leader, sending a full tablet report...
I20260808 05:01:01.002678 29592 heartbeater.cc:503] Master 127.3.174.190:38397 was elected leader, sending a full tablet report...
I20260808 05:01:01.029667 29618 raft_consensus.cc:482] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:01.029806 29618 raft_consensus.cc:504] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:01.030128 29618 leader_election.cc:288] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775), 86a4c7a5f81b4e5e846d9155bb3d247a (127.3.174.129:33889)
I20260808 05:01:01.033891 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.030401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:59903 (local address 127.3.174.129:33889)
0808 05:01:01.030446 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:01.030448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:01.030580 (+ 132us) server_negotiation.cc:407] Connection header received
0808 05:01:01.030617 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:01.030619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:01.030646 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:01.030697 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:01.030842 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.031574 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.032928 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.033573 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.033639 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:01.033701 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:01.033775 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 05:01:01.033817 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 05:01:01.033893 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.030371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:38577 (local address 127.3.174.130:38775)
0808 05:01:01.030415 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:01.030417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:01.030449 (+ 32us) server_negotiation.cc:407] Connection header received
0808 05:01:01.030529 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:01.030529 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:01.030539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:01.030591 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:01.030747 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.031498 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.032926 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.033524 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.033595 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:01.033606 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:01.033776 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 05:01:01.033816 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 05:01:01.033910 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.030302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:38775 (local address 127.3.174.131:38577)
0808 05:01:01.030396 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.030409 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:01.030466 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:01.030613 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:01.030614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:01.030616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:01.030684 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:01.030685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.031598 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.031599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:01.032833 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:01.032836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.033603 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.033604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:01.033701 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:01.033708 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 05:01:01.033760 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 05:01:01.033808 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":12}
I20260808 05:01:01.033918 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.030330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33889 (local address 127.3.174.131:59903)
0808 05:01:01.030503 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.030513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:01.030571 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:01.030714 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:01.030715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:01.030717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:01.030782 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:01.030783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.031657 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.031657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:01.032833 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:01.032836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.033640 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.033640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:01.033701 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:01.033708 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 05:01:01.033765 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 05:01:01.033808 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":4,"thread_start_us":119,"threads_started":1}
I20260808 05:01:01.034336 29273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4736c170a5da4547993f4989b880f4d1" candidate_uuid: "b622d8824a2e486b945735e7888edf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" is_pre_election: true
I20260808 05:01:01.034512 29273 raft_consensus.cc:2454] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b622d8824a2e486b945735e7888edf48 in term 0.
I20260808 05:01:01.034492 29014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4736c170a5da4547993f4989b880f4d1" candidate_uuid: "b622d8824a2e486b945735e7888edf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" is_pre_election: true
I20260808 05:01:01.034647 29014 raft_consensus.cc:2454] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b622d8824a2e486b945735e7888edf48 in term 0.
I20260808 05:01:01.034873 29395 leader_election.cc:302] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86a4c7a5f81b4e5e846d9155bb3d247a, b622d8824a2e486b945735e7888edf48; no voters:
I20260808 05:01:01.035014 29618 raft_consensus.cc:2781] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:01:01.035076 29618 raft_consensus.cc:482] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:01:01.035094 29618 raft_consensus.cc:3037] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:01.035828 29618 raft_consensus.cc:504] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:01.035972 29618 leader_election.cc:288] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [CANDIDATE]: Term 1 election: Requested vote from peers 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775), 86a4c7a5f81b4e5e846d9155bb3d247a (127.3.174.129:33889)
I20260808 05:01:01.036180 29273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4736c170a5da4547993f4989b880f4d1" candidate_uuid: "b622d8824a2e486b945735e7888edf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a82bd4dbe8545a9985fd1563c118ad9"
I20260808 05:01:01.036208 29014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4736c170a5da4547993f4989b880f4d1" candidate_uuid: "b622d8824a2e486b945735e7888edf48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a"
I20260808 05:01:01.036258 29273 raft_consensus.cc:3037] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:01.036279 29014 raft_consensus.cc:3037] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:01.036963 29014 raft_consensus.cc:2454] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b622d8824a2e486b945735e7888edf48 in term 1.
I20260808 05:01:01.036972 29273 raft_consensus.cc:2454] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b622d8824a2e486b945735e7888edf48 in term 1.
I20260808 05:01:01.037133 29390 leader_election.cc:302] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a82bd4dbe8545a9985fd1563c118ad9, b622d8824a2e486b945735e7888edf48; no voters:
I20260808 05:01:01.037233 29618 raft_consensus.cc:2781] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:01:01.037462 29618 raft_consensus.cc:686] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 1 LEADER]: Becoming Leader. State: Replica: b622d8824a2e486b945735e7888edf48, State: Running, Role: LEADER
I20260808 05:01:01.037550 29618 consensus_queue.cc:237] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:01.038398 28757 catalog_manager.cc:5495] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 reported cstate change: term changed from 0 to 1, leader changed from <none> to b622d8824a2e486b945735e7888edf48 (127.3.174.131). New cstate: current_term: 1 leader_uuid: "b622d8824a2e486b945735e7888edf48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } health_report { overall_health: UNKNOWN } } }
I20260808 05:01:01.054661 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.052352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.3.174.131:42595)
0808 05:01:01.052399 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:01.052401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:01.052410 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:01:01.052490 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:01.052491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:01.052510 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:01.052570 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:01:01.052691 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.053182 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.053935 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:01.054132 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:01.054186 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:01.054334 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:01:01.054436 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:01:01.054507 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:01.054545 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 05:01:01.054566 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 05:01:01.056480 29273 raft_consensus.cc:1261] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [term 1 FOLLOWER]: Refusing update from remote peer b622d8824a2e486b945735e7888edf48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:01:01.056507 29014 raft_consensus.cc:1261] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Refusing update from remote peer b622d8824a2e486b945735e7888edf48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:01:01.056836 29618 consensus_queue.cc:1035] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:01:01.056972 29620 consensus_queue.cc:1035] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:01:01.061342 29626 mvcc.cc:205] Tried to move back new op lower bound from 7316132909284745216 to 7316132909210517504. Current Snapshot: MvccSnapshot[applied={T|T < 7316132909284745216 or (T in {7316132909284745216})}]
I20260808 05:01:01.061512 28758 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:01:01.061606 28758 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:01:01.062304 28758 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Sending DeleteTablet for 3 replicas of tablet 4736c170a5da4547993f4989b880f4d1
I20260808 05:01:01.062628 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 28820
I20260808 05:01:01.062705 28981 tablet_service.cc:1514] Processing DeleteTablet for tablet 4736c170a5da4547993f4989b880f4d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:01 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:55606
I20260808 05:01:01.062819 29234 tablet_service.cc:1514] Processing DeleteTablet for tablet 4736c170a5da4547993f4989b880f4d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:01 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:35380
I20260808 05:01:01.062999 29470 tablet_service.cc:1514] Processing DeleteTablet for tablet 4736c170a5da4547993f4989b880f4d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:01 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:60242
I20260808 05:01:01.079015 29636 tablet_replica.cc:330] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: stopping tablet replica
I20260808 05:01:01.079200 29636 raft_consensus.cc:2229] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:01:01.079352 29636 raft_consensus.cc:2258] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:01.083731 29639 tablet_replica.cc:330] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: stopping tablet replica
I20260808 05:01:01.084017 29639 raft_consensus.cc:2229] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:01:01.084245 29639 raft_consensus.cc:2258] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:01.084942 29636 ts_tablet_manager.cc:1905] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:01.086782 29636 ts_tablet_manager.cc:1918] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:01.086871 29636 log.cc:1196] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/4736c170a5da4547993f4989b880f4d1
I20260808 05:01:01.086781 29639 ts_tablet_manager.cc:1905] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:01.087153 29636 ts_tablet_manager.cc:1939] T 4736c170a5da4547993f4989b880f4d1 P 4a82bd4dbe8545a9985fd1563c118ad9: Deleting consensus metadata
I20260808 05:01:01.087659 28730 catalog_manager.cc:4841] TS 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775): tablet 4736c170a5da4547993f4989b880f4d1 (table test-table [id=648ee178711d4606835d393c8c841232]) successfully deleted
I20260808 05:01:01.089129 29639 ts_tablet_manager.cc:1918] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:01.089233 29639 log.cc:1196] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/4736c170a5da4547993f4989b880f4d1
I20260808 05:01:01.089634 29639 ts_tablet_manager.cc:1939] T 4736c170a5da4547993f4989b880f4d1 P b622d8824a2e486b945735e7888edf48: Deleting consensus metadata
I20260808 05:01:01.090263 28731 catalog_manager.cc:4841] TS b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595): tablet 4736c170a5da4547993f4989b880f4d1 (table test-table [id=648ee178711d4606835d393c8c841232]) successfully deleted
W20260808 05:01:01.105214 28737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:33889 (error 108)
I20260808 05:01:01.105378 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 29108
I20260808 05:01:01.109755 29602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.109625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33889 (local address 127.0.0.1:55616)
0808 05:01:01.109682 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.109721 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 05:01:01.109941 28737 catalog_manager.cc:4584] TS 86a4c7a5f81b4e5e846d9155bb3d247a (127.3.174.129:33889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4736c170a5da4547993f4989b880f4d1: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33889: connect: Connection refused (error 111)
I20260808 05:01:01.114218 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 29362
I20260808 05:01:01.123574 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 28697
I20260808 05:01:01.135401 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38397
--webserver_interface=127.3.174.190
--webserver_port=43373
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:38397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:01.219357 29641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:01.219594 29641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:01.219624 29641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:01.220764 29641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:01:01.220803 29641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:01.220822 29641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:01.220839 29641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:01:01.220855 29641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:01:01.222396 29641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:38397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=43373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:01.222641 29641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:01.222851 29641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:01.227574 29648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:01.227574 29656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:01.227646 29650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:01.227836 29641 server_base.cc:1034] running on GCE node
I20260808 05:01:01.228109 29641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:01.228358 29641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:01.229496 29641 hybrid_clock.cc:648] HybridClock initialized: now 1786165261229485 us; error 29 us; skew 500 ppm
Aug 08 05:01:01 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165261, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:01.231956 29641 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:01:01.232393 29641 webserver.cc:460] Webserver started at http://127.3.174.190:43373/ using document root <none> and password file <none>
I20260808 05:01:01.232614 29641 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:01.232662 29641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:01.234000 29641 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260808 05:01:01.234607 29666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:01.234761 29641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:01:01.234848 29641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "c8382fe5abf14d84b548d57ce2782f47"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "f03fc049ce996a218aded5b33df96428"
server_key_iv: "68f3d06f42bb4dfd2751dd3e72f67df7"
server_key_version: "encryptionkey@0"
I20260808 05:01:01.235181 29641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:01.245615 29641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:01.247400 29641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:01.247526 29641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:01.251560 29641 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:38397
I20260808 05:01:01.251606 29764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:38397 every 8 connection(s)
I20260808 05:01:01.251961 29641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:01:01.253584 29765 sys_catalog.cc:263] Verifying existing consensus state
I20260808 05:01:01.254231 29765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Bootstrap starting.
I20260808 05:01:01.260630 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 29641
I20260808 05:01:01.260987 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:33889
--local_ip_for_outbound_sockets=127.3.174.129
--tserver_master_addrs=127.3.174.190:38397
--webserver_port=32863
--webserver_interface=127.3.174.129
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:01:01.263682 29765 log.cc:824] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:01.266206 29765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 05:01:01.266486 29765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47: Bootstrap complete.
I20260808 05:01:01.268680 29765 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:01:01.268918 29765 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8382fe5abf14d84b548d57ce2782f47, State: Initialized, Role: FOLLOWER
I20260808 05:01:01.269071 29765 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:01:01.269132 29765 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:01:01.269171 29765 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:01:01.269217 29765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 2 FOLLOWER]: Advancing to term 3
I20260808 05:01:01.270249 29765 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:01:01.270345 29765 leader_election.cc:302] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8382fe5abf14d84b548d57ce2782f47; no voters:
I20260808 05:01:01.270525 29765 leader_election.cc:288] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 05:01:01.270579 29775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 05:01:01.270695 29775 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [term 3 LEADER]: Becoming Leader. State: Replica: c8382fe5abf14d84b548d57ce2782f47, State: Running, Role: LEADER
I20260808 05:01:01.270795 29775 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } }
I20260808 05:01:01.270826 29765 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:01:01.271117 29776 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c8382fe5abf14d84b548d57ce2782f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } } }
I20260808 05:01:01.271360 29776 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:01.271127 29775 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8382fe5abf14d84b548d57ce2782f47. Latest consensus state: current_term: 3 leader_uuid: "c8382fe5abf14d84b548d57ce2782f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8382fe5abf14d84b548d57ce2782f47" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38397 } } }
I20260808 05:01:01.271634 29775 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8382fe5abf14d84b548d57ce2782f47 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:01.271739 29790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:01:01.272739 29790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:01:01.272852 29790 catalog_manager.cc:1259] Loaded cluster ID: 7475274bf4f141bdab7302cb2eab0f92
I20260808 05:01:01.272886 29790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 05:01:01.273276 29790 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb40a6379ad google::LogMessage::Fail() at ??:0
@ 0x7fb40a63a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb40a6374de google::LogMessage::Flush() at ??:0
@ 0x7fb40a638a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb4125580f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb4125618b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb40c5050d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb40c4fa954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb40d35e6db start_thread at ??:0
@ 0x7fb40913561f clone at ??:0
W20260808 05:01:01.357683 29770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:01.357898 29770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:01.357933 29770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:01.359086 29770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:01.359125 29770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:01.359159 29770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:01:01.360617 29770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:33889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=32863
--tserver_master_addrs=127.3.174.190:38397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:01.360858 29770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:01.361070 29770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:01.366111 29828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:01.366367 29770 server_base.cc:1034] running on GCE node
W20260808 05:01:01.366511 29832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:01.366144 29829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:01.366801 29770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:01.367094 29770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:01.368275 29770 hybrid_clock.cc:648] HybridClock initialized: now 1786165261368251 us; error 40 us; skew 500 ppm
Aug 08 05:01:01 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165261, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:01.370918 29770 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:01:01.371413 29770 webserver.cc:460] Webserver started at http://127.3.174.129:32863/ using document root <none> and password file <none>
I20260808 05:01:01.371634 29770 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:01.371698 29770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:01.373140 29770 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:01.373937 29848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:01.374104 29770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 05:01:01.374197 29770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "86a4c7a5f81b4e5e846d9155bb3d247a"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "37a2007e55725bc24fbd47c7fad27133"
server_key_iv: "d469d3d7d670a62605d59b61be4b4eaf"
server_key_version: "encryptionkey@0"
I20260808 05:01:01.374570 29770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:01.382057 29770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:01.384196 29770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:01.384342 29770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:01.384560 29770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:01.385039 29858 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 05:01:01.387039 29770 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 05:01:01.387099 29770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 05:01:01.387136 29770 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 05:01:01.387872 29770 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 05:01:01.387912 29770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 05:01:01.388002 29858 tablet_bootstrap.cc:492] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Bootstrap starting.
I20260808 05:01:01.395536 29770 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:33889
I20260808 05:01:01.395606 30073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:33889 every 8 connection(s)
I20260808 05:01:01.396052 29770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:01:01.397599 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 29770
I20260808 05:01:01.397930 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:38775
--local_ip_for_outbound_sockets=127.3.174.130
--tserver_master_addrs=127.3.174.190:38397
--webserver_port=43773
--webserver_interface=127.3.174.130
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:01:01.399982 29858 log.cc:824] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:01.402973 29858 tablet_bootstrap.cc:492] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 05:01:01.403288 29858 tablet_bootstrap.cc:492] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Bootstrap complete.
I20260808 05:01:01.404572 29858 ts_tablet_manager.cc:1397] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.011s
I20260808 05:01:01.406139 29858 raft_consensus.cc:348] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:01.406524 29858 raft_consensus.cc:729] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a4c7a5f81b4e5e846d9155bb3d247a, State: Initialized, Role: FOLLOWER
I20260808 05:01:01.406765 29858 consensus_queue.cc:260] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:01.407087 29858 ts_tablet_manager.cc:1428] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260808 05:01:01.466746 30078 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:01:01.396824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:54623)
0808 05:01:01.397029 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.397077 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:01.397749 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:01.466559 (+ 68810us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
W20260808 05:01:01.467049 30075 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:38397 (0 consecutive failures): Network error: Failed to ping master at 127.3.174.190:38397: Client connection negotiation failed: client connection to 127.3.174.190:38397: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 05:01:01.467439 30078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.467317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:52001)
0808 05:01:01.467372 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.467415 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 05:01:01.467813 30078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.467731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:54383)
0808 05:01:01.467773 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.467796 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 05:01:01.467931 30075 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 05:01:01.488662 30081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:01.488894 30081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:01.488919 30081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:01.490126 30081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:01.490170 30081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:01.490202 30081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:01:01.491667 30081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:38775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=43773
--tserver_master_addrs=127.3.174.190:38397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:01.491921 30081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:01.492123 30081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:01.497320 30095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:01.497227 30100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:01.497393 30081 server_base.cc:1034] running on GCE node
W20260808 05:01:01.497519 30094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:01.497710 30081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:01.497958 30081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:01.499117 30081 hybrid_clock.cc:648] HybridClock initialized: now 1786165261499109 us; error 35 us; skew 500 ppm
Aug 08 05:01:01 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165261, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:01.501860 30081 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:01:01.502297 30081 webserver.cc:460] Webserver started at http://127.3.174.130:43773/ using document root <none> and password file <none>
I20260808 05:01:01.502485 30081 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:01.502524 30081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:01.503908 30081 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 05:01:01.504565 30111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:01.504729 30081 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 05:01:01.504810 30081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "4a82bd4dbe8545a9985fd1563c118ad9"
format_stamp: "Formatted at 2026-08-08 05:00:59 on dist-test-slave-gjw7"
server_key: "b9d364445cd91484d9b8badc1ed4ce4b"
server_key_iv: "0ca61c2f0951929fe772c32ad2329c16"
server_key_version: "encryptionkey@0"
I20260808 05:01:01.505210 30081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:01.520458 30081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:01.522975 30081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:01.523141 30081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:01.523420 30081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:01.523765 30081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:01.523805 30081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:01.523840 30081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:01.523869 30081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:01.530493 30081 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:38775
I20260808 05:01:01.530560 30346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:38775 every 8 connection(s)
I20260808 05:01:01.530938 30081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 05:01:01.532049 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.531657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:53087)
0808 05:01:01.531812 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.531899 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
W20260808 05:01:01.532472 30347 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:38397 (0 consecutive failures): Network error: Failed to ping master at 127.3.174.190:38397: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
I20260808 05:01:01.532759 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.532672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:42315)
0808 05:01:01.532713 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.532742 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 05:01:01.533062 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.533008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:48207)
0808 05:01:01.533031 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.533049 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 05:01:01.533147 30347 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 05:01:01.534389 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 30081
I20260808 05:01:01.534615 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:42595
--local_ip_for_outbound_sockets=127.3.174.131
--tserver_master_addrs=127.3.174.190:38397
--webserver_port=35795
--webserver_interface=127.3.174.131
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:01.620349 30353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:01.620550 30353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:01.620584 30353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:01.621747 30353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:01.621785 30353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:01.621820 30353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:01:01.623229 30353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:42595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=35795
--tserver_master_addrs=127.3.174.190:38397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:01.623473 30353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:01.623685 30353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:01.628568 30364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:01.628549 30366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:01.628558 30360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:01.628885 30353 server_base.cc:1034] running on GCE node
I20260808 05:01:01.629051 30353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:01.629243 30353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:01.630415 30353 hybrid_clock.cc:648] HybridClock initialized: now 1786165261630399 us; error 30 us; skew 500 ppm
Aug 08 05:01:01 dist-test-slave-gjw7 krb5kdc[27723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165261, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:01.632805 30353 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:01:01.633244 30353 webserver.cc:460] Webserver started at http://127.3.174.131:35795/ using document root <none> and password file <none>
I20260808 05:01:01.633483 30353 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:01.633536 30353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:01.634876 30353 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:01.635488 30380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:01.635634 30353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:01:01.635721 30353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "b622d8824a2e486b945735e7888edf48"
format_stamp: "Formatted at 2026-08-08 05:01:00 on dist-test-slave-gjw7"
server_key: "51ab7717fefbe38ded48dab6c7cafedf"
server_key_iv: "b3a7e3a002c83705cacba259734eb371"
server_key_version: "encryptionkey@0"
I20260808 05:01:01.636061 30353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:01.647418 30353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:01.649374 30353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:01.649503 30353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:01.649712 30353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:01.650002 30353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:01.650035 30353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:01.650063 30353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:01.650086 30353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:01.655704 30353 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:42595
I20260808 05:01:01.655778 30597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:42595 every 8 connection(s)
I20260808 05:01:01.656100 30353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:01:01.657081 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.656726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:60057)
0808 05:01:01.656871 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.656939 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
W20260808 05:01:01.657529 30601 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:38397 (0 consecutive failures): Network error: Failed to ping master at 127.3.174.190:38397: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
I20260808 05:01:01.657786 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.657704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:57233)
0808 05:01:01.657743 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.657769 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 05:01:01.657994 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:01.657946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:37693)
0808 05:01:01.657965 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:01.657983 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260808 05:01:01.658072 30601 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 05:01:01.660284 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 30353
I20260808 05:01:02.468611 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:02.468347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:39439)
0808 05:01:02.468504 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:02.468544 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260808 05:01:02.533619 30610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:02.533433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:42975)
0808 05:01:02.533539 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:02.533572 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260808 05:01:02.658579 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:02.658375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:52057)
0808 05:01:02.658495 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:02.658529 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260808 05:01:02.741492 30613 raft_consensus.cc:482] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:02.741623 30613 raft_consensus.cc:504] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:02.742010 30613 leader_election.cc:288] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595), 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775)
Aug 08 05:01:02 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165261, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.130@KRBTEST.COM
Aug 08 05:01:02 dist-test-slave-gjw7 krb5kdc[27723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165261, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.131@KRBTEST.COM
I20260808 05:01:02.751616 30616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:02.742334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:60327 (local address 127.3.174.130:38775)
0808 05:01:02.742495 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:02.742504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:02.742535 (+ 31us) server_negotiation.cc:407] Connection header received
0808 05:01:02.742978 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:02.742980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:02.743029 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:02.743159 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:02.743912 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:02.744781 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:02.745746 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:02.746134 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:02.746237 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:02.748373 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:02.748377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:02.748377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:02.748389 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:02.750074 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:02.750498 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:02.750501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:02.750501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:02.750590 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:02.750813 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:02.750814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:02.750814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:02.751033 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:02.751105 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:02.751390 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 05:01:02.751506 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260808 05:01:02.751622 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:02.742342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:47245 (local address 127.3.174.131:42595)
0808 05:01:02.742480 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:02.742491 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:02.742517 (+ 26us) server_negotiation.cc:407] Connection header received
0808 05:01:02.742973 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:02.742976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:02.743018 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:02.743147 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:02.743911 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:02.744771 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:02.745751 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:02.746135 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:02.746234 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:02.748739 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:02.748743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:02.748744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:02.748755 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:02.750314 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:02.750619 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:02.750622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:02.750622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:02.750703 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:02.750882 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:02.750883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:02.750883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:02.751092 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:02.751160 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:02.751389 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 05:01:02.751503 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
I20260808 05:01:02.751634 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:02.742151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:42595 (local address 127.3.174.129:47245)
0808 05:01:02.742254 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:02.742268 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:02.742334 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:02.743196 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:02.743207 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:02.743604 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:02.743789 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:02.743795 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:02.744869 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:02.744870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:02.745661 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:02.745664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:02.746218 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:02.746219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:02.746324 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:02.746883 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:02.746888 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:02.748617 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:02.750394 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:02.750395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:02.750396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:02.750552 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:02.750773 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:02.750774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:02.750775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:02.750834 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:02.751157 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:02.751158 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:02.751243 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 05:01:02.751328 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 05:01:02.751507 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":64}
I20260808 05:01:02.752035 30614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:02.742156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:38775 (local address 127.3.174.129:60327)
0808 05:01:02.742304 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:02.742312 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:02.742363 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:02.743197 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:02.743200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:02.743607 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:02.743789 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:02.743795 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:02.744875 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:02.744876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:02.745660 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:02.745663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:02.746213 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:02.746214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:02.746322 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:02.746842 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:02.746853 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:02.748273 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:02.750148 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:02.750155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:02.750157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:02.750386 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:02.750650 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:02.750651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:02.750651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:02.750758 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:02.751097 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:02.751104 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:02.751238 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 05:01:02.751328 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 05:01:02.751577 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":143,"thread_start_us":76,"threads_started":1}
W20260808 05:01:02.752403 29851 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
W20260808 05:01:02.752678 29850 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
I20260808 05:01:02.752741 29850 leader_election.cc:302] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86a4c7a5f81b4e5e846d9155bb3d247a; no voters: 4a82bd4dbe8545a9985fd1563c118ad9, b622d8824a2e486b945735e7888edf48
I20260808 05:01:02.752841 30613 raft_consensus.cc:2726] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 05:01:03.469329 30617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:03.469065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:57171)
0808 05:01:03.469206 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:03.469244 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260808 05:01:03.534307 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:03.534095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:41701)
0808 05:01:03.534220 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:03.534265 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260808 05:01:03.659268 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:03.659002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:37711)
0808 05:01:03.659157 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:03.659197 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260808 05:01:04.470084 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:04.469866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:49035)
0808 05:01:04.469997 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:04.470036 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260808 05:01:04.534981 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:04.534733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:35487)
0808 05:01:04.534885 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:04.534937 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260808 05:01:04.638115 30623 raft_consensus.cc:482] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:04.638247 30623 raft_consensus.cc:504] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:04.638469 30623 leader_election.cc:288] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595), 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775)
W20260808 05:01:04.638865 29850 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
W20260808 05:01:04.638985 29851 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
I20260808 05:01:04.639050 29851 leader_election.cc:302] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86a4c7a5f81b4e5e846d9155bb3d247a; no voters: 4a82bd4dbe8545a9985fd1563c118ad9, b622d8824a2e486b945735e7888edf48
I20260808 05:01:04.639120 30623 raft_consensus.cc:2726] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 05:01:04.659888 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:04.659692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:48007)
0808 05:01:04.659811 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:04.659846 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260808 05:01:05.470805 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:05.470594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:33565)
0808 05:01:05.470720 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:05.470756 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260808 05:01:05.535633 30627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:05.535441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:53489)
0808 05:01:05.535555 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:05.535592 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260808 05:01:05.660573 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:05.660367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:33139)
0808 05:01:05.660495 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:05.660532 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260808 05:01:06.471653 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:06.471405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:37189)
0808 05:01:06.471557 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:06.471604 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260808 05:01:06.536301 30634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:06.536101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:39805)
0808 05:01:06.536207 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:06.536244 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260808 05:01:06.661407 30635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:06.661074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:38673)
0808 05:01:06.661312 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:06.661361 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":84,"threads_started":1}
I20260808 05:01:06.762786 30637 raft_consensus.cc:482] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:06.762898 30637 raft_consensus.cc:504] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:06.763069 30637 leader_election.cc:288] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595), 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775)
W20260808 05:01:06.763465 29851 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
W20260808 05:01:06.763566 29850 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
I20260808 05:01:06.763625 29850 leader_election.cc:302] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86a4c7a5f81b4e5e846d9155bb3d247a; no voters: 4a82bd4dbe8545a9985fd1563c118ad9, b622d8824a2e486b945735e7888edf48
I20260808 05:01:06.763716 30637 raft_consensus.cc:2726] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 05:01:07.472440 30638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:07.472166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:42435)
0808 05:01:07.472350 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:07.472389 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
I20260808 05:01:07.537039 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:07.536802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:60873)
0808 05:01:07.536943 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:07.536994 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260808 05:01:07.662081 30643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:07.661874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:52789)
0808 05:01:07.661999 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:07.662035 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260808 05:01:08.473119 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:08.472904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:40865)
0808 05:01:08.473032 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:08.473069 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260808 05:01:08.537662 30646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:08.537456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:60829)
0808 05:01:08.537586 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:08.537619 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260808 05:01:08.662701 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:08.662496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:48787)
0808 05:01:08.662616 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:08.662651 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260808 05:01:09.114418 30649 raft_consensus.cc:482] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:09.114521 30649 raft_consensus.cc:504] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:09.114699 30649 leader_election.cc:288] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595), 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775)
W20260808 05:01:09.115154 29850 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
W20260808 05:01:09.115252 29851 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
I20260808 05:01:09.115299 29851 leader_election.cc:302] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86a4c7a5f81b4e5e846d9155bb3d247a; no voters: 4a82bd4dbe8545a9985fd1563c118ad9, b622d8824a2e486b945735e7888edf48
I20260808 05:01:09.115373 30649 raft_consensus.cc:2726] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 05:01:09.473832 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:09.473600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:49683)
0808 05:01:09.473738 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:09.473780 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":66,"threads_started":1}
I20260808 05:01:09.538328 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:09.538079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:48025)
0808 05:01:09.538242 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:09.538282 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260808 05:01:09.663324 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:09.663114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:51029)
0808 05:01:09.663242 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:09.663280 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260808 05:01:10.474547 30656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:10.474311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:52599)
0808 05:01:10.474456 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:10.474497 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260808 05:01:10.538962 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:10.538752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:55741)
0808 05:01:10.538882 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:10.538917 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260808 05:01:10.663925 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:10.663729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:49891)
0808 05:01:10.663845 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:10.663881 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260808 05:01:11.475456 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:11.475134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:60931)
0808 05:01:11.475357 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:11.475407 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":132,"threads_started":1}
I20260808 05:01:11.539784 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:11.539520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:54007)
0808 05:01:11.539671 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:11.539730 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260808 05:01:11.664664 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:11.664418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:51301)
0808 05:01:11.664574 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:11.664617 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260808 05:01:12.476292 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:12.476023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:42469)
0808 05:01:12.476182 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:12.476233 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260808 05:01:12.540537 30670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:12.540282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:55717)
0808 05:01:12.540441 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:12.540487 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260808 05:01:12.665391 30672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:12.665093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:44565)
0808 05:01:12.665258 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:12.665341 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260808 05:01:13.477080 30673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:13.476796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:53147)
0808 05:01:13.476977 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:13.477020 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":93,"threads_started":1}
I20260808 05:01:13.541275 30675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:13.541017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:44065)
0808 05:01:13.541174 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:13.541217 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260808 05:01:13.666078 30677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:13.665847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:41259)
0808 05:01:13.665990 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:13.666031 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260808 05:01:14.186239 30678 raft_consensus.cc:482] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:14.186353 30678 raft_consensus.cc:504] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b622d8824a2e486b945735e7888edf48" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42595 } } peers { permanent_uuid: "4a82bd4dbe8545a9985fd1563c118ad9" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38775 } } peers { permanent_uuid: "86a4c7a5f81b4e5e846d9155bb3d247a" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33889 } }
I20260808 05:01:14.186534 30678 leader_election.cc:288] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595), 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775)
W20260808 05:01:14.186959 29850 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a82bd4dbe8545a9985fd1563c118ad9 (127.3.174.130:38775): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
W20260808 05:01:14.187055 29851 leader_election.cc:341] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b622d8824a2e486b945735e7888edf48 (127.3.174.131:42595): Not found: Tablet not found: 4736c170a5da4547993f4989b880f4d1
I20260808 05:01:14.187131 29851 leader_election.cc:302] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86a4c7a5f81b4e5e846d9155bb3d247a; no voters: 4a82bd4dbe8545a9985fd1563c118ad9, b622d8824a2e486b945735e7888edf48
I20260808 05:01:14.187202 30678 raft_consensus.cc:2726] T 4736c170a5da4547993f4989b880f4d1 P 86a4c7a5f81b4e5e846d9155bb3d247a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 05:01:14.477830 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:14.477567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:38759)
0808 05:01:14.477718 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:14.477762 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260808 05:01:14.542012 30683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:14.541769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:54629)
0808 05:01:14.541902 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:14.541944 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260808 05:01:14.666734 30685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:14.666484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:58249)
0808 05:01:14.666639 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:14.666684 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
I20260808 05:01:15.478523 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:15.478296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:39117)
0808 05:01:15.478425 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:15.478465 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260808 05:01:15.542712 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:15.542477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:39363)
0808 05:01:15.542614 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:15.542652 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260808 05:01:15.667528 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:15.667260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:51289)
0808 05:01:15.667431 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:15.667478 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260808 05:01:16.479303 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:16.479052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.129:38341)
0808 05:01:16.479206 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:16.479250 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260808 05:01:16.543466 30692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:16.543258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.130:36051)
0808 05:01:16.543381 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:16.543420 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260808 05:01:16.667184 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 29770
I20260808 05:01:16.702925 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:16.701750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:38397 (local address 127.3.174.131:42277)
0808 05:01:16.702806 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:16.702861 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1029,"thread_start_us":950,"threads_started":1}
I20260808 05:01:16.708143 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 30081
I20260808 05:01:16.719311 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 30353
2026-08-08T05:01:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18535 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:01:16 dist-test-slave-gjw7 krb5kdc[30699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:01:16 dist-test-slave-gjw7 krb5kdc[30699](info): set up 2 sockets
Aug 08 05:01:16 dist-test-slave-gjw7 krb5kdc[30699](info): commencing operation
krb5kdc: starting...
W20260808 05:01:17.903431 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:01:17 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:01:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:01:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:01:18.024295 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45123
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:45123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:18.107115 30723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:18.107326 30723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:18.107347 30723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:18.108433 30723 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 05:01:18.108462 30723 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 05:01:18.108474 30723 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 05:01:18.108484 30723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:01:18.108496 30723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:18.108511 30723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:18.108522 30723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:01:18.108538 30723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:01:18.109972 30723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:45123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:18.110167 30723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:18.110349 30723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 05:01:18.115221 30723 server_base.cc:1034] running on GCE node
W20260808 05:01:18.115350 30736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:18.115343 30731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:18.115343 30729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:18.115830 30723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:18.116115 30723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:18.117314 30723 hybrid_clock.cc:648] HybridClock initialized: now 1786165278117268 us; error 33 us; skew 500 ppm
Aug 08 05:01:18 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165278, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:18.119863 30723 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:01:18.120316 30723 webserver.cc:460] Webserver started at http://127.3.174.190:43933/ using document root <none> and password file <none>
I20260808 05:01:18.120517 30723 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:18.120568 30723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:18.120694 30723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:01:18.121778 30723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "b2c53df135dd4d10b95f69d28ca24863"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "5a3b04675a5254c8d864f62c93da4aaf"
server_key_iv: "ee96dec75be68fd86369aead0a321cb1"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.122113 30723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "b2c53df135dd4d10b95f69d28ca24863"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "5a3b04675a5254c8d864f62c93da4aaf"
server_key_iv: "ee96dec75be68fd86369aead0a321cb1"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.123400 30723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 05:01:18.124100 30752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.124260 30723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:01:18.124347 30723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "b2c53df135dd4d10b95f69d28ca24863"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "5a3b04675a5254c8d864f62c93da4aaf"
server_key_iv: "ee96dec75be68fd86369aead0a321cb1"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.124411 30723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:18.150020 30723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:18.151826 30723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:18.151959 30723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:18.155900 30723 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:45123
I20260808 05:01:18.155954 30846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:45123 every 8 connection(s)
I20260808 05:01:18.156432 30723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:01:18.157097 30847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:18.159461 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 30723
I20260808 05:01:18.159583 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:01:18.159596 30847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Bootstrap starting.
I20260808 05:01:18.159806 3770 external_mini_cluster.cc:1428] Setting key 70112e4d70787ee2f24edc06b9f06085
I20260808 05:01:18.160295 30847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:18.160658 30847 log.cc:824] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:18.161324 30847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: No bootstrap required, opened a new log
I20260808 05:01:18.163189 30847 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:18.163467 30847 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:18.163537 30847 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c53df135dd4d10b95f69d28ca24863, State: Initialized, Role: FOLLOWER
I20260808 05:01:18.163693 30847 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:18.163779 30847 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:01:18.163820 30847 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:01:18.163872 30847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:18.164556 30847 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:18.164688 30847 leader_election.cc:302] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2c53df135dd4d10b95f69d28ca24863; no voters:
I20260808 05:01:18.164875 30847 leader_election.cc:288] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:01:18.164906 30854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:01:18.165030 30854 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 1 LEADER]: Becoming Leader. State: Replica: b2c53df135dd4d10b95f69d28ca24863, State: Running, Role: LEADER
I20260808 05:01:18.165221 30847 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:01:18.165200 30854 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:18.165587 30854 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2c53df135dd4d10b95f69d28ca24863. Latest consensus state: current_term: 1 leader_uuid: "b2c53df135dd4d10b95f69d28ca24863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } } }
I20260808 05:01:18.165858 30854 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:18.165776 30855 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2c53df135dd4d10b95f69d28ca24863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } } }
I20260808 05:01:18.166361 30865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 05:01:18 dist-test-slave-gjw7 krb5kdc[30699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:18.166393 30855 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:18.167057 30865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:01:18.169003 30865 catalog_manager.cc:1347] Generated new cluster ID: 36919a836f2e438e81a988f9b612bf14
I20260808 05:01:18.169054 30865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:01:18.170470 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.160342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33424 (local address 127.3.174.190:45123)
0808 05:01:18.160519 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.160528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.160555 (+ 27us) server_negotiation.cc:407] Connection header received
0808 05:01:18.160998 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.161007 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.161058 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.161186 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:18.161907 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.162775 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.163526 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.164085 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.164184 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.167044 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:18.167049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:18.167049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:18.167062 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:18.168796 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.169188 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.169191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.169191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.169321 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.169504 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.169505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.169505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.169777 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:18.169893 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.170126 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.170269 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260808 05:01:18.183478 30865 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:01:18.183983 30865 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:01:18.193372 30865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Generated new TSK 0
I20260808 05:01:18.193533 30865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 05:01:18.219697 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45123
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:18.300937 30887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:18.301134 30887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:18.301155 30887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:18.302263 30887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:18.302299 30887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:18.302322 30887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:01:18.303728 30887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:18.303922 30887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:18.304092 30887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:18.308746 30896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:18.308737 30892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:18.308876 30887 server_base.cc:1034] running on GCE node
W20260808 05:01:18.308959 30893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:18.309194 30887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:18.309428 30887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:18.310556 30887 hybrid_clock.cc:648] HybridClock initialized: now 1786165278310545 us; error 28 us; skew 500 ppm
Aug 08 05:01:18 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165278, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:18.312959 30887 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:01:18.313395 30887 webserver.cc:460] Webserver started at http://127.3.174.129:36041/ using document root <none> and password file <none>
I20260808 05:01:18.313589 30887 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:18.313632 30887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:18.313756 30887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:01:18.314729 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "14afd3da8e144fd792cec0d10c77f5e4"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "b38fd9046c25234a76bd77c30b86cd7d"
server_key_iv: "14daae5fbe2025eb0c936efc7e1094c0"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.315058 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "14afd3da8e144fd792cec0d10c77f5e4"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "b38fd9046c25234a76bd77c30b86cd7d"
server_key_iv: "14daae5fbe2025eb0c936efc7e1094c0"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.316277 30887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:01:18.316896 30914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.317027 30887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:01:18.317113 30887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "14afd3da8e144fd792cec0d10c77f5e4"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "b38fd9046c25234a76bd77c30b86cd7d"
server_key_iv: "14daae5fbe2025eb0c936efc7e1094c0"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.317174 30887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:18.334532 30887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:18.336580 30887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:18.336709 30887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:18.336922 30887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:18.337250 30887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:18.337335 30887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.337374 30887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:18.337393 30887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.343235 30887 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:39783
I20260808 05:01:18.343325 31125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:39783 every 8 connection(s)
I20260808 05:01:18.343654 30887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:01:18.344404 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 30887
I20260808 05:01:18.344521 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:01:18.344746 3770 external_mini_cluster.cc:1428] Setting key 99a5f32e460f09605c975de921ace757
Aug 08 05:01:18 dist-test-slave-gjw7 krb5kdc[30699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165278, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:18.353360 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.344399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:36767 (local address 127.3.174.190:45123)
0808 05:01:18.344445 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.344447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.344721 (+ 274us) server_negotiation.cc:407] Connection header received
0808 05:01:18.345374 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.345375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.345387 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.345431 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:18.346379 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.347071 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.348484 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.348686 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.348746 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.350796 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:18.350798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:18.350798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:18.350805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:18.352088 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.352442 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.352443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.352443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.352499 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.352708 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.352709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.352709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.352819 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:18.352874 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.353124 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.353216 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 05:01:18.353490 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.344389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:36767)
0808 05:01:18.344575 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.344623 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.345247 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.345505 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.345509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.346081 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:18.346282 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.346288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.347214 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.347215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.348402 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.348406 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.348777 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.348779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.348893 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.349373 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:18.349382 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:18.350703 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:18.352161 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:18.352169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:18.352171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:18.352375 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:18.352553 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:18.352553 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:18.352554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:18.352654 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:18.352874 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:18.352881 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:18.353011 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.353096 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.353249 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260808 05:01:18.353947 31127 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45123
I20260808 05:01:18.354043 31127 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
I20260808 05:01:18.354212 31127 heartbeater.cc:511] Master 127.3.174.190:45123 requested a full tablet report, sending...
I20260808 05:01:18.354755 30783 ts_manager.cc:194] Registered new tserver with Master: 14afd3da8e144fd792cec0d10c77f5e4 (127.3.174.129:39783)
I20260808 05:01:18.355388 30783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:36767
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 05:01:18.371261 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45123
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:18.452481 31132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:18.452680 31132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:18.452711 31132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:18.453964 31132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:18.454010 31132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:18.454044 31132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:01:18.455492 31132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:18.455735 31132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:18.455958 31132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:18.460743 31146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:18.460813 31140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:18.460817 31143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:18.460847 31132 server_base.cc:1034] running on GCE node
I20260808 05:01:18.461215 31132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:18.461475 31132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:18.462611 31132 hybrid_clock.cc:648] HybridClock initialized: now 1786165278462599 us; error 33 us; skew 500 ppm
Aug 08 05:01:18 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165278, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:18.465050 31132 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:01:18.465560 31132 webserver.cc:460] Webserver started at http://127.3.174.130:33557/ using document root <none> and password file <none>
I20260808 05:01:18.465777 31132 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:18.465826 31132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:18.465960 31132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:01:18.467057 31132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "c897bcef3ca24482b9e2d4e4c99ac99c"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "c6ddc990b9eb317fbacab52e2292439d"
server_key_iv: "7a9030f6368a89f5bd387d90f65fd4e6"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.467417 31132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "c897bcef3ca24482b9e2d4e4c99ac99c"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "c6ddc990b9eb317fbacab52e2292439d"
server_key_iv: "7a9030f6368a89f5bd387d90f65fd4e6"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.468761 31132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:01:18.469519 31158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.469667 31132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:01:18.469756 31132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "c897bcef3ca24482b9e2d4e4c99ac99c"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "c6ddc990b9eb317fbacab52e2292439d"
server_key_iv: "7a9030f6368a89f5bd387d90f65fd4e6"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.469837 31132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:18.488981 31132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:18.490764 31132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:18.490909 31132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:18.491132 31132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:18.491464 31132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:18.491503 31132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.491533 31132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:18.491560 31132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.497197 31132 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:40343
I20260808 05:01:18.497262 31363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:40343 every 8 connection(s)
I20260808 05:01:18.497576 31132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 05:01:18 dist-test-slave-gjw7 krb5kdc[30699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165278, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:18.505874 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 31132
I20260808 05:01:18.505983 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:01:18.506102 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.498299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:56601 (local address 127.3.174.190:45123)
0808 05:01:18.498361 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.498363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.498529 (+ 166us) server_negotiation.cc:407] Connection header received
0808 05:01:18.499020 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.499021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.499031 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.499084 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:18.499974 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.500476 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.501424 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.501604 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.501650 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.503545 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:18.503547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:18.503547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:18.503554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:18.504797 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.505176 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.505177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.505177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.505223 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.505482 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.505483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.505483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.505592 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:18.505643 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.505919 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.506008 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 05:01:18.506187 3770 external_mini_cluster.cc:1428] Setting key ecf7e3ba93c11b5590e09f0408b869b7
I20260808 05:01:18.506237 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.498249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:56601)
0808 05:01:18.498411 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.498452 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.498946 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.499140 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.499144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.499696 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:18.499910 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.499915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.500548 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.500549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.501362 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.501364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.501663 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.501663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.501760 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.502157 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:18.502165 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:18.503450 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:18.504881 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:18.504889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:18.504891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:18.505113 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:18.505270 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:18.505271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:18.505271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:18.505420 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:18.505646 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:18.505652 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:18.505778 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.505870 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.506023 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260808 05:01:18.506798 31365 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45123
I20260808 05:01:18.506893 31365 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:18.507066 31365 heartbeater.cc:511] Master 127.3.174.190:45123 requested a full tablet report, sending...
I20260808 05:01:18.507470 30783 ts_manager.cc:194] Registered new tserver with Master: c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
I20260808 05:01:18.507836 30783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:56601
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 05:01:18.530145 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45123
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:18.612941 31376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:18.613142 31376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:18.613163 31376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:18.614332 31376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:18.614375 31376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:18.614400 31376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:01:18.615846 31376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:18.616077 31376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:18.616291 31376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:18.621212 31385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:18.621356 31376 server_base.cc:1034] running on GCE node
W20260808 05:01:18.621203 31386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:18.621204 31389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:18.621654 31376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:18.621862 31376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:18.623008 31376 hybrid_clock.cc:648] HybridClock initialized: now 1786165278622994 us; error 29 us; skew 500 ppm
Aug 08 05:01:18 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165278, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:18.625550 31376 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:01:18.625972 31376 webserver.cc:460] Webserver started at http://127.3.174.131:36421/ using document root <none> and password file <none>
I20260808 05:01:18.626204 31376 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:18.626262 31376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:18.626387 31376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:01:18.627430 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "01bb19fd75124364b9982a61157afe4b"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "bdb697f5f7067ba1c2f78f001b8ba82a"
server_key_iv: "8804c060236e14587473f0a4f52f8740"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.627763 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "01bb19fd75124364b9982a61157afe4b"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "bdb697f5f7067ba1c2f78f001b8ba82a"
server_key_iv: "8804c060236e14587473f0a4f52f8740"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.629086 31376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:01:18.629786 31401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.629940 31376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.630016 31376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "01bb19fd75124364b9982a61157afe4b"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "bdb697f5f7067ba1c2f78f001b8ba82a"
server_key_iv: "8804c060236e14587473f0a4f52f8740"
server_key_version: "encryptionkey@0"
I20260808 05:01:18.630079 31376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:18.652654 31376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:18.654948 31376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:18.655093 31376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:18.655341 31376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:18.655644 31376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:18.655678 31376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.655709 31376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:18.655738 31376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:18.661258 31376 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:41869
I20260808 05:01:18.661368 31613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:41869 every 8 connection(s)
I20260808 05:01:18.661638 31376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:01:18.665199 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 31376
I20260808 05:01:18.665344 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 05:01:18.665544 3770 external_mini_cluster.cc:1428] Setting key 979cbddfdd2c518be8dda52a31a18200
Aug 08 05:01:18 dist-test-slave-gjw7 krb5kdc[30699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165278, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:18.670653 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.662327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:39323 (local address 127.3.174.190:45123)
0808 05:01:18.662381 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.662383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.662568 (+ 185us) server_negotiation.cc:407] Connection header received
0808 05:01:18.663067 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.663069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.663081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.663131 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:18.664088 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.664633 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.666001 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.666210 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.666269 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.668381 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:18.668382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:18.668383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:18.668390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:18.669640 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.669964 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.669964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.669965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.670009 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.670179 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.670179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.670179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.670282 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:18.670330 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.670529 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.670586 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 05:01:18.670876 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.662311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:39323)
0808 05:01:18.662454 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.662492 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.662973 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.663202 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.663206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.663791 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:18.663992 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.663997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.664725 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.664726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.665867 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.665870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.666283 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.666284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.666422 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.666904 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:18.666913 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:18.668285 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:18.669697 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:18.669701 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:18.669703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:18.669911 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:18.670051 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:18.670051 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:18.670052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:18.670139 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:18.670326 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:18.670329 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:18.670441 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.670517 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.670633 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260808 05:01:18.671351 31616 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45123
I20260808 05:01:18.671443 31616 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:18.671622 31616 heartbeater.cc:511] Master 127.3.174.190:45123 requested a full tablet report, sending...
I20260808 05:01:18.671983 30783 ts_manager.cc:194] Registered new tserver with Master: 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869)
I20260808 05:01:18.672317 30783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:39323
I20260808 05:01:18.677129 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:01:18.683364 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.678421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.3.174.190:45123)
0808 05:01:18.678482 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.678484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.678530 (+ 46us) server_negotiation.cc:407] Connection header received
0808 05:01:18.678576 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.678577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.678585 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.678626 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:18.679093 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.679547 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.680185 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.680353 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.680393 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.681398 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:18.681399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:18.681399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:18.681405 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:18.682558 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.682817 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.682818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.682818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.682862 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:18.682970 (+ 108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:18.682971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:18.682971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:18.683096 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:18.683165 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.683237 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.683301 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 05:01:18.684758 30783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:01:18.686484 30783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:01:18.692363 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.688465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39783 (local address 127.0.0.1:40334)
0808 05:01:18.688595 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.688635 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.688701 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.689183 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.689187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.689190 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:18.689261 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.689264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.690296 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.690297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.691124 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.691126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.692055 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.692056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.692146 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.692162 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.692235 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.692283 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":4,"thread_start_us":82,"threads_started":1}
I20260808 05:01:18.692397 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.688759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40334 (local address 127.3.174.129:39783)
0808 05:01:18.688927 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.688932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.688945 (+ 13us) server_negotiation.cc:407] Connection header received
0808 05:01:18.688990 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.688992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.689009 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.689120 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:18.689434 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.690198 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.691211 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.691973 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.692047 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.692108 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.692257 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.692301 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260808 05:01:18.692822 31629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.688470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:41869 (local address 127.0.0.1:49414)
0808 05:01:18.689059 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.689068 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.689115 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.689377 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.689378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.689381 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:18.689425 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.689426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.690294 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.690296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.691530 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.691533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.692520 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.692521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.692651 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.692656 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.692712 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.692750 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"mutex_wait_us":41,"thread_start_us":59,"threads_started":1}
I20260808 05:01:18.692886 31632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.688726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.3.174.131:41869)
0808 05:01:18.688929 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.688934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.689116 (+ 182us) server_negotiation.cc:407] Connection header received
0808 05:01:18.689191 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.689194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.689210 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.689259 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:18.689482 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.690197 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.691611 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.692283 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.692344 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.692399 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.692745 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.692802 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":103,"threads_started":1}
I20260808 05:01:18.693225 31003 tablet_service.cc:1467] Processing CreateTablet for tablet f4754684a03949d9b803999dded41cec (DEFAULT_TABLE table=test-table [id=4816ed6e02ab4931a0b6312a597749ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:18.693447 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.688470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:40343 (local address 127.0.0.1:51404)
0808 05:01:18.688932 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.688942 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.688996 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.689253 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.689254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.689257 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:18.689343 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.689346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.691545 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.691546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.692377 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.692380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.693208 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.693209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.693269 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.693273 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.693350 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.693386 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":61,"threads_started":1}
I20260808 05:01:18.693681 31003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4754684a03949d9b803999dded41cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:18.693748 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.688948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.3.174.130:40343)
0808 05:01:18.689099 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.689103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.689113 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:01:18.689152 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.689154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.689170 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.689254 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:18.690337 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.691435 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.692466 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.693133 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.693190 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.693226 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.693623 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.693668 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260808 05:01:18.696242 31636 tablet_bootstrap.cc:492] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Bootstrap starting.
I20260808 05:01:18.696656 31501 tablet_service.cc:1467] Processing CreateTablet for tablet f4754684a03949d9b803999dded41cec (DEFAULT_TABLE table=test-table [id=4816ed6e02ab4931a0b6312a597749ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:18.696957 31501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4754684a03949d9b803999dded41cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:18.697057 31636 tablet_bootstrap.cc:654] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:18.697587 31636 log.cc:824] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:18.698247 31636 tablet_bootstrap.cc:492] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: No bootstrap required, opened a new log
I20260808 05:01:18.698313 31636 ts_tablet_manager.cc:1397] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 05:01:18.699416 31639 tablet_bootstrap.cc:492] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: Bootstrap starting.
I20260808 05:01:18.699831 31253 tablet_service.cc:1467] Processing CreateTablet for tablet f4754684a03949d9b803999dded41cec (DEFAULT_TABLE table=test-table [id=4816ed6e02ab4931a0b6312a597749ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:18.699745 31636 raft_consensus.cc:348] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.700002 31636 raft_consensus.cc:374] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:18.700055 31636 raft_consensus.cc:729] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14afd3da8e144fd792cec0d10c77f5e4, State: Initialized, Role: FOLLOWER
I20260808 05:01:18.700165 31253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4754684a03949d9b803999dded41cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:18.700259 31639 tablet_bootstrap.cc:654] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:18.700201 31636 consensus_queue.cc:260] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.700505 31636 ts_tablet_manager.cc:1428] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 05:01:18.700548 31127 heartbeater.cc:503] Master 127.3.174.190:45123 was elected leader, sending a full tablet report...
I20260808 05:01:18.701162 31639 log.cc:824] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:18.702134 31639 tablet_bootstrap.cc:492] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: No bootstrap required, opened a new log
I20260808 05:01:18.702229 31639 ts_tablet_manager.cc:1397] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 05:01:18.703437 31646 tablet_bootstrap.cc:492] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: Bootstrap starting.
I20260808 05:01:18.703991 31646 tablet_bootstrap.cc:654] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:18.703979 31639 raft_consensus.cc:348] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.704156 31639 raft_consensus.cc:374] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:18.704203 31639 raft_consensus.cc:729] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bb19fd75124364b9982a61157afe4b, State: Initialized, Role: FOLLOWER
I20260808 05:01:18.704321 31639 consensus_queue.cc:260] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.704537 31646 log.cc:824] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:18.704598 31639 ts_tablet_manager.cc:1428] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 05:01:18.704661 31616 heartbeater.cc:503] Master 127.3.174.190:45123 was elected leader, sending a full tablet report...
I20260808 05:01:18.705366 31646 tablet_bootstrap.cc:492] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: No bootstrap required, opened a new log
I20260808 05:01:18.705440 31646 ts_tablet_manager.cc:1397] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:01:18.706801 31646 raft_consensus.cc:348] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.706938 31646 raft_consensus.cc:374] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:18.706969 31646 raft_consensus.cc:729] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c897bcef3ca24482b9e2d4e4c99ac99c, State: Initialized, Role: FOLLOWER
I20260808 05:01:18.707077 31646 consensus_queue.cc:260] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.707311 31646 ts_tablet_manager.cc:1428] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:01:18.707350 31365 heartbeater.cc:503] Master 127.3.174.190:45123 was elected leader, sending a full tablet report...
I20260808 05:01:18.744877 31647 raft_consensus.cc:482] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:18.745026 31647 raft_consensus.cc:504] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.745428 31647 leader_election.cc:288] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14afd3da8e144fd792cec0d10c77f5e4 (127.3.174.129:39783), c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
W20260808 05:01:18.748090 31366 tablet.cc:2369] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:01:18.749173 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.745664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:43191 (local address 127.3.174.129:39783)
0808 05:01:18.745707 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.745709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.745735 (+ 26us) server_negotiation.cc:407] Connection header received
0808 05:01:18.745765 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.745766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.745790 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.745839 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:18.746020 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.746761 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.748226 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.748831 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.748904 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.748912 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.749057 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.749103 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 05:01:18.749176 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.745581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39783 (local address 127.3.174.131:43191)
0808 05:01:18.745653 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.745663 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.745702 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.745872 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.745873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.745875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:18.745950 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.745952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.746850 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.746850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.748121 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.748124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.748907 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.748908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.748982 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.748987 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.749032 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.749071 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":53}
I20260808 05:01:18.749188 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.745846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:55517 (local address 127.3.174.130:40343)
0808 05:01:18.745888 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.745890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.745897 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:01:18.745920 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.745921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.745928 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.745971 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:18.746099 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.746789 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.748253 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.748841 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.748884 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.748894 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.749090 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.749137 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 05:01:18.749199 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.745593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:40343 (local address 127.3.174.131:55517)
0808 05:01:18.745806 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.745816 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.745874 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.745987 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.745988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.745990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:18.746044 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.746045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.746853 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.746853 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.748121 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.748124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.748955 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.748956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.749017 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.749020 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.749070 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.749109 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":37,"thread_start_us":125,"threads_started":1}
I20260808 05:01:18.749424 31654 raft_consensus.cc:482] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:18.749506 31654 raft_consensus.cc:504] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.749588 31043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4754684a03949d9b803999dded41cec" candidate_uuid: "01bb19fd75124364b9982a61157afe4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14afd3da8e144fd792cec0d10c77f5e4" is_pre_election: true
I20260808 05:01:18.749684 31288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4754684a03949d9b803999dded41cec" candidate_uuid: "01bb19fd75124364b9982a61157afe4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" is_pre_election: true
I20260808 05:01:18.749750 31043 raft_consensus.cc:2454] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bb19fd75124364b9982a61157afe4b in term 0.
I20260808 05:01:18.749869 31288 raft_consensus.cc:2454] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bb19fd75124364b9982a61157afe4b in term 0.
I20260808 05:01:18.750006 31404 leader_election.cc:302] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bb19fd75124364b9982a61157afe4b, 14afd3da8e144fd792cec0d10c77f5e4; no voters:
I20260808 05:01:18.750337 31647 raft_consensus.cc:2781] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:01:18.750413 31647 raft_consensus.cc:482] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:01:18.750440 31647 raft_consensus.cc:3037] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:18.750633 31654 leader_election.cc:288] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14afd3da8e144fd792cec0d10c77f5e4 (127.3.174.129:39783), 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869)
I20260808 05:01:18.751220 31647 raft_consensus.cc:504] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.751332 31647 leader_election.cc:288] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [CANDIDATE]: Term 1 election: Requested vote from peers 14afd3da8e144fd792cec0d10c77f5e4 (127.3.174.129:39783), c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
I20260808 05:01:18.751489 31288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4754684a03949d9b803999dded41cec" candidate_uuid: "01bb19fd75124364b9982a61157afe4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c"
I20260808 05:01:18.751478 31043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4754684a03949d9b803999dded41cec" candidate_uuid: "01bb19fd75124364b9982a61157afe4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14afd3da8e144fd792cec0d10c77f5e4"
I20260808 05:01:18.751552 31288 raft_consensus.cc:3037] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:18.751554 31043 raft_consensus.cc:3037] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:18.752395 31288 raft_consensus.cc:2454] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bb19fd75124364b9982a61157afe4b in term 1.
I20260808 05:01:18.752395 31043 raft_consensus.cc:2454] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bb19fd75124364b9982a61157afe4b in term 1.
I20260808 05:01:18.752589 31404 leader_election.cc:302] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 01bb19fd75124364b9982a61157afe4b, 14afd3da8e144fd792cec0d10c77f5e4, c897bcef3ca24482b9e2d4e4c99ac99c; no voters:
I20260808 05:01:18.752701 31647 raft_consensus.cc:2781] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:01:18.752846 31647 raft_consensus.cc:686] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 1 LEADER]: Becoming Leader. State: Replica: 01bb19fd75124364b9982a61157afe4b, State: Running, Role: LEADER
I20260808 05:01:18.752940 31647 consensus_queue.cc:237] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:18.753222 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.750173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:41869 (local address 127.3.174.130:39707)
0808 05:01:18.750221 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.750230 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.750265 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.750497 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.750498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.750500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:18.750555 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.750556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.751484 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.751486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.752238 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.752240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.752997 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.752998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.753077 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.753081 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.753123 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.753158 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 05:01:18.753512 31632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.750209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:39707 (local address 127.3.174.131:41869)
0808 05:01:18.750371 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.750372 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.750379 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:01:18.750399 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.750399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.750407 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.750457 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:18.750627 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.751318 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.752331 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.752850 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.752916 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.752924 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.753420 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.753458 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260808 05:01:18.753707 31538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4754684a03949d9b803999dded41cec" candidate_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bb19fd75124364b9982a61157afe4b" is_pre_election: true
I20260808 05:01:18.753741 30783 catalog_manager.cc:5495] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bb19fd75124364b9982a61157afe4b (127.3.174.131). New cstate: current_term: 1 leader_uuid: "01bb19fd75124364b9982a61157afe4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } health_report { overall_health: HEALTHY } } }
I20260808 05:01:18.754300 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.750236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39783 (local address 127.3.174.130:40929)
0808 05:01:18.750376 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.750386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:18.750459 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:18.750611 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:18.750612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:18.750613 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:18.750673 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.750675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.752339 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.752340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.753393 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:18.753396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.754049 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.754050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:18.754137 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.754142 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:01:18.754194 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 05:01:18.754228 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260808 05:01:18.754374 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.750277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:40929 (local address 127.3.174.129:39783)
0808 05:01:18.750321 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.750324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.750435 (+ 111us) server_negotiation.cc:407] Connection header received
0808 05:01:18.750524 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.750526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.750534 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.750559 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:18.750757 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.751407 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.753496 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.753974 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.754070 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.754078 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.754283 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.754311 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 05:01:18.754534 31043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4754684a03949d9b803999dded41cec" candidate_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14afd3da8e144fd792cec0d10c77f5e4" is_pre_election: true
I20260808 05:01:18.754629 31043 raft_consensus.cc:2379] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c897bcef3ca24482b9e2d4e4c99ac99c in current term 1: Already voted for candidate 01bb19fd75124364b9982a61157afe4b in this term.
I20260808 05:01:18.754827 31160 leader_election.cc:302] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c897bcef3ca24482b9e2d4e4c99ac99c; no voters: 01bb19fd75124364b9982a61157afe4b, 14afd3da8e144fd792cec0d10c77f5e4
I20260808 05:01:18.754945 31654 raft_consensus.cc:2726] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 05:01:18.764155 31632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.762262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49416 (local address 127.3.174.131:41869)
0808 05:01:18.762309 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:18.762311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:18.762321 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:01:18.762387 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:18.762388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:18.762403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:18.762457 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:01:18.762539 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.762947 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.763617 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:18.763779 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:18.763821 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:18.763912 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:01:18.764002 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:01:18.764053 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:18.764064 (+ 11us) server_negotiation.cc:299] Negotiation successful
0808 05:01:18.764077 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 05:01:18.765749 31288 raft_consensus.cc:1261] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 1 FOLLOWER]: Refusing update from remote peer 01bb19fd75124364b9982a61157afe4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:01:18.765753 31043 raft_consensus.cc:1261] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Refusing update from remote peer 01bb19fd75124364b9982a61157afe4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:01:18.766018 31647 consensus_queue.cc:1035] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:01:18.766224 31665 consensus_queue.cc:1035] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:01:18.770627 30781 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:01:18.770709 30781 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:01:18.771400 30781 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Sending DeleteTablet for 3 replicas of tablet f4754684a03949d9b803999dded41cec
I20260808 05:01:18.771744 31501 tablet_service.cc:1514] Processing DeleteTablet for tablet f4754684a03949d9b803999dded41cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:18 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:49414
I20260808 05:01:18.771795 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 30887
I20260808 05:01:18.771781 31003 tablet_service.cc:1514] Processing DeleteTablet for tablet f4754684a03949d9b803999dded41cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:18 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:40334
I20260808 05:01:18.778741 31253 tablet_service.cc:1514] Processing DeleteTablet for tablet f4754684a03949d9b803999dded41cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:18 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:51404
I20260808 05:01:18.781656 31697 tablet_replica.cc:330] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: stopping tablet replica
I20260808 05:01:18.781775 31697 raft_consensus.cc:2229] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:01:18.781877 31697 raft_consensus.cc:2258] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:18.784317 31697 ts_tablet_manager.cc:1905] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:18.786069 31695 tablet_replica.cc:330] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: stopping tablet replica
I20260808 05:01:18.786092 31697 ts_tablet_manager.cc:1918] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:18.786228 31695 raft_consensus.cc:2229] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:01:18.786360 31697 log.cc:1196] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/f4754684a03949d9b803999dded41cec
I20260808 05:01:18.786399 31695 raft_consensus.cc:2258] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:18.786767 31697 ts_tablet_manager.cc:1939] T f4754684a03949d9b803999dded41cec P c897bcef3ca24482b9e2d4e4c99ac99c: Deleting consensus metadata
I20260808 05:01:18.787285 30756 catalog_manager.cc:4841] TS c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343): tablet f4754684a03949d9b803999dded41cec (table test-table [id=4816ed6e02ab4931a0b6312a597749ef]) successfully deleted
I20260808 05:01:18.787354 31695 ts_tablet_manager.cc:1905] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:18.788831 31695 ts_tablet_manager.cc:1918] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:18.788905 31695 log.cc:1196] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/f4754684a03949d9b803999dded41cec
I20260808 05:01:18.789160 31695 ts_tablet_manager.cc:1939] T f4754684a03949d9b803999dded41cec P 01bb19fd75124364b9982a61157afe4b: Deleting consensus metadata
I20260808 05:01:18.789618 30754 catalog_manager.cc:4841] TS 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869): tablet f4754684a03949d9b803999dded41cec (table test-table [id=4816ed6e02ab4931a0b6312a597749ef]) successfully deleted
W20260808 05:01:18.791767 30753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:39783 (error 108)
I20260808 05:01:18.792249 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:18.792124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39783 (local address 127.0.0.1:40340)
0808 05:01:18.792178 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:18.792216 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:39783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 05:01:18.792359 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 31132
W20260808 05:01:18.805321 30753 catalog_manager.cc:4584] TS 14afd3da8e144fd792cec0d10c77f5e4 (127.3.174.129:39783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4754684a03949d9b803999dded41cec: Network error: Client connection negotiation failed: client connection to 127.3.174.129:39783: connect: Connection refused (error 111)
I20260808 05:01:18.811599 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 31376
I20260808 05:01:18.825837 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 30723
I20260808 05:01:38.835307 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45123
--webserver_interface=127.3.174.190
--webserver_port=43933
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:45123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:38.922614 31699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:38.922868 31699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:38.922892 31699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:38.924146 31699 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 05:01:38.924196 31699 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 05:01:38.924209 31699 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 05:01:38.924220 31699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:01:38.924232 31699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:38.924245 31699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:38.924260 31699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:01:38.924289 31699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:01:38.925953 31699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:45123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=43933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:38.926232 31699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:38.926486 31699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:38.936857 31710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:38.936904 31699 server_base.cc:1034] running on GCE node
W20260808 05:01:38.936902 31712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:38.936869 31715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:38.937493 31699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:38.937836 31699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:38.938988 31699 hybrid_clock.cc:648] HybridClock initialized: now 1786165298938969 us; error 33 us; skew 500 ppm
Aug 08 05:01:38 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165298, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:38.945762 31699 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:01:38.946314 31699 webserver.cc:460] Webserver started at http://127.3.174.190:43933/ using document root <none> and password file <none>
I20260808 05:01:38.946556 31699 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:38.946623 31699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:38.948124 31699 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:38.948841 31730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:38.949107 31699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:38.949227 31699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "b2c53df135dd4d10b95f69d28ca24863"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "5a3b04675a5254c8d864f62c93da4aaf"
server_key_iv: "ee96dec75be68fd86369aead0a321cb1"
server_key_version: "encryptionkey@0"
I20260808 05:01:38.949703 31699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:38.960848 31699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:38.963193 31699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:38.963346 31699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:38.967700 31699 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:45123
I20260808 05:01:38.967751 31823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:45123 every 8 connection(s)
I20260808 05:01:38.968163 31699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:01:38.969803 31824 sys_catalog.cc:263] Verifying existing consensus state
I20260808 05:01:38.970498 31824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Bootstrap starting.
I20260808 05:01:38.971923 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 31699
I20260808 05:01:38.972254 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:39783
--local_ip_for_outbound_sockets=127.3.174.129
--tserver_master_addrs=127.3.174.190:45123
--webserver_port=36041
--webserver_interface=127.3.174.129
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:01:38.980041 31824 log.cc:824] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:38.982246 31824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 05:01:38.982571 31824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Bootstrap complete.
I20260808 05:01:38.984828 31824 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:38.985074 31824 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c53df135dd4d10b95f69d28ca24863, State: Initialized, Role: FOLLOWER
I20260808 05:01:38.985231 31824 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:38.985323 31824 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:01:38.985368 31824 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:01:38.985417 31824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 1 FOLLOWER]: Advancing to term 2
I20260808 05:01:38.986505 31824 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:38.986608 31824 leader_election.cc:302] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2c53df135dd4d10b95f69d28ca24863; no voters:
I20260808 05:01:38.986801 31824 leader_election.cc:288] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 05:01:38.986860 31828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 05:01:38.987046 31828 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 2 LEADER]: Becoming Leader. State: Replica: b2c53df135dd4d10b95f69d28ca24863, State: Running, Role: LEADER
I20260808 05:01:38.987102 31824 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:01:38.987159 31828 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:38.987462 31828 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2c53df135dd4d10b95f69d28ca24863. Latest consensus state: current_term: 2 leader_uuid: "b2c53df135dd4d10b95f69d28ca24863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } } }
I20260808 05:01:38.987630 31828 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:38.987577 31832 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b2c53df135dd4d10b95f69d28ca24863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } } }
I20260808 05:01:38.988040 31832 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:38.988260 31841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:01:38.989164 31841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:01:38.989310 31841 catalog_manager.cc:1259] Loaded cluster ID: 36919a836f2e438e81a988f9b612bf14
I20260808 05:01:38.989346 31841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:01:38.990593 31841 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:01:38.990809 31841 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Loaded TSK: 0
I20260808 05:01:39.003970 31841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Generated new TSK 1
I20260808 05:01:39.004108 31841 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 05:01:39.062822 31826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:39.063079 31826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:39.063117 31826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:39.064579 31826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:39.064667 31826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:39.064718 31826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:01:39.066867 31826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:39783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=36041
--tserver_master_addrs=127.3.174.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:39.067304 31826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:39.067582 31826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:39.073355 31878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:39.073453 31874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:39.073462 31872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:39.073791 31826 server_base.cc:1034] running on GCE node
I20260808 05:01:39.074004 31826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:39.074266 31826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:39.075424 31826 hybrid_clock.cc:648] HybridClock initialized: now 1786165299075404 us; error 36 us; skew 500 ppm
Aug 08 05:01:39 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165299, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:39.078886 31826 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:01:39.079521 31826 webserver.cc:460] Webserver started at http://127.3.174.129:36041/ using document root <none> and password file <none>
I20260808 05:01:39.079819 31826 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:39.079912 31826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:39.081936 31826 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 05:01:39.082840 31890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:39.083042 31826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:39.083158 31826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "14afd3da8e144fd792cec0d10c77f5e4"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "b38fd9046c25234a76bd77c30b86cd7d"
server_key_iv: "14daae5fbe2025eb0c936efc7e1094c0"
server_key_version: "encryptionkey@0"
I20260808 05:01:39.083667 31826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:39.099774 31826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:39.102653 31826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:39.102828 31826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:39.103121 31826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:39.103669 31908 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 05:01:39.105350 31826 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 05:01:39.105415 31826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 05:01:39.105453 31826 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 05:01:39.106379 31826 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 05:01:39.106429 31826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 05:01:39.106523 31908 tablet_bootstrap.cc:492] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Bootstrap starting.
I20260808 05:01:39.114113 31826 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:39783
I20260808 05:01:39.114311 32095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:39783 every 8 connection(s)
I20260808 05:01:39.114629 31826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:01:39.117522 31908 log.cc:824] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:39.119045 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 31826
I20260808 05:01:39.119375 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:40343
--local_ip_for_outbound_sockets=127.3.174.130
--tserver_master_addrs=127.3.174.190:45123
--webserver_port=33557
--webserver_interface=127.3.174.130
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:01:39.120426 31908 tablet_bootstrap.cc:492] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 05:01:39.120746 31908 tablet_bootstrap.cc:492] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Bootstrap complete.
I20260808 05:01:39.122088 31908 ts_tablet_manager.cc:1397] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.013s
I20260808 05:01:39.123489 31908 raft_consensus.cc:348] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:39.123795 31908 raft_consensus.cc:729] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14afd3da8e144fd792cec0d10c77f5e4, State: Initialized, Role: FOLLOWER
I20260808 05:01:39.123924 31908 consensus_queue.cc:260] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } }
I20260808 05:01:39.124204 31908 ts_tablet_manager.cc:1428] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Aug 08 05:01:39 dist-test-slave-gjw7 krb5kdc[30699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165299, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:39.131354 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.115436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:37607)
0808 05:01:39.117421 (+ 1985us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:39.117471 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:39.118294 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:39.118749 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:39.118753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:39.119252 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:39.119771 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.119781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.121580 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.121581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.122884 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.122888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.123778 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.123779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.124140 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.124850 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:39.124862 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:39.126757 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:39.129228 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:39.129236 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:39.129239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:39.129608 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:39.129943 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:39.129944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:39.129945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:39.130102 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:39.130636 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:39.130644 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:39.130808 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 05:01:39.130917 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 05:01:39.131101 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1930,"thread_start_us":94,"threads_started":1}
I20260808 05:01:39.131732 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.115547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:37607 (local address 127.3.174.190:45123)
0808 05:01:39.115766 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.115776 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.117684 (+ 1908us) server_negotiation.cc:407] Connection header received
0808 05:01:39.118532 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.118538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.118587 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.118707 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:39.119915 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.121424 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.122984 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.123568 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.123649 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.126898 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:39.126904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:39.126904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:39.126921 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:39.129118 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.129719 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.129723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.129724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.129864 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.130184 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.130185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.130185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.130549 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:39.130647 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.131024 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.131182 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":115,"threads_started":1}
I20260808 05:01:39.132469 32096 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45123
I20260808 05:01:39.132583 32096 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:39.132782 32096 heartbeater.cc:511] Master 127.3.174.190:45123 requested a full tablet report, sending...
I20260808 05:01:39.133658 31759 ts_manager.cc:194] Registered new tserver with Master: 14afd3da8e144fd792cec0d10c77f5e4 (127.3.174.129:39783)
I20260808 05:01:39.134886 31759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:37607
I20260808 05:01:39.135964 32096 heartbeater.cc:503] Master 127.3.174.190:45123 was elected leader, sending a full tablet report...
I20260808 05:01:39.141573 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.134358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39783 (local address 127.0.0.1:47980)
0808 05:01:39.137513 (+ 3155us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:39.137557 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:39.137629 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:39.137868 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:39.137872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:39.137876 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:39.137958 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.137960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.139204 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.139205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.140325 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.140328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.141260 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.141262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.141391 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.141404 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 05:01:39.141467 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 05:01:39.141510 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3122,"thread_start_us":100,"threads_started":1}
I20260808 05:01:39.141633 32112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.134629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47980 (local address 127.3.174.129:39783)
0808 05:01:39.137435 (+ 2806us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.137441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.137704 (+ 263us) server_negotiation.cc:407] Connection header received
0808 05:01:39.137741 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.137743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.137758 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.137829 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:39.138033 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.139110 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.140413 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.141171 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.141242 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.141336 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.141495 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.141541 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2764,"thread_start_us":114,"threads_started":1}
I20260808 05:01:39.142023 31986 tablet_service.cc:1514] Processing DeleteTablet for tablet f4754684a03949d9b803999dded41cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:18 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:47980
I20260808 05:01:39.142652 32113 tablet_replica.cc:330] stopping tablet replica
I20260808 05:01:39.142771 32113 raft_consensus.cc:2229] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:01:39.142854 32113 raft_consensus.cc:2258] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:39.143177 32113 ts_tablet_manager.cc:1905] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:39.145485 32113 ts_tablet_manager.cc:1918] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:39.145570 32113 log.cc:1196] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal/wals/f4754684a03949d9b803999dded41cec
I20260808 05:01:39.145771 32113 ts_tablet_manager.cc:1939] T f4754684a03949d9b803999dded41cec P 14afd3da8e144fd792cec0d10c77f5e4: Deleting consensus metadata
I20260808 05:01:39.146319 31731 catalog_manager.cc:4841] TS 14afd3da8e144fd792cec0d10c77f5e4 (127.3.174.129:39783): tablet f4754684a03949d9b803999dded41cec (table test-table [id=4816ed6e02ab4931a0b6312a597749ef]) successfully deleted
W20260808 05:01:39.212250 32109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:39.212488 32109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:39.212524 32109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:39.213789 32109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:39.213835 32109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:39.213871 32109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:01:39.215405 32109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:40343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=33557
--tserver_master_addrs=127.3.174.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:39.215659 32109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:39.215922 32109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:39.221007 32126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:39.221004 32121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:39.221164 32109 server_base.cc:1034] running on GCE node
W20260808 05:01:39.221004 32123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:39.221462 32109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:39.221709 32109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:39.222877 32109 hybrid_clock.cc:648] HybridClock initialized: now 1786165299222857 us; error 29 us; skew 500 ppm
Aug 08 05:01:39 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165299, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:39.225749 32109 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:01:39.226270 32109 webserver.cc:460] Webserver started at http://127.3.174.130:33557/ using document root <none> and password file <none>
I20260808 05:01:39.226495 32109 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:39.226564 32109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:39.227982 32109 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:39.228559 32139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:39.228734 32109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:39.228825 32109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "c897bcef3ca24482b9e2d4e4c99ac99c"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "c6ddc990b9eb317fbacab52e2292439d"
server_key_iv: "7a9030f6368a89f5bd387d90f65fd4e6"
server_key_version: "encryptionkey@0"
I20260808 05:01:39.229202 32109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:39.241758 32109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:39.243955 32109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:39.244104 32109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:39.244359 32109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:39.244684 32109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:39.244725 32109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:39.244760 32109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:39.244783 32109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:39.250617 32109 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:40343
I20260808 05:01:39.250686 32359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:40343 every 8 connection(s)
I20260808 05:01:39.251041 32109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 05:01:39.256659 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 32109
I20260808 05:01:39.256982 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:41869
--local_ip_for_outbound_sockets=127.3.174.131
--tserver_master_addrs=127.3.174.190:45123
--webserver_port=36421
--webserver_interface=127.3.174.131
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 05:01:39 dist-test-slave-gjw7 krb5kdc[30699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165299, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:39.261041 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.251820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:33111 (local address 127.3.174.190:45123)
0808 05:01:39.251887 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.251889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.252110 (+ 221us) server_negotiation.cc:407] Connection header received
0808 05:01:39.252603 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.252604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.252620 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.252679 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:39.253836 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.254352 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.255363 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.255569 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.255630 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.258142 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:39.258144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:39.258144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:39.258154 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:39.259656 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.260073 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.260074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.260074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.260132 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.260371 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.260372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.260373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.260529 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:39.260599 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.260846 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.260946 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 05:01:39.261132 32364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.251790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:33111)
0808 05:01:39.251965 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:39.252006 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:39.252514 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:39.252750 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:39.252753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:39.253521 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:39.253743 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.253749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.254443 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.254444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.255269 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.255272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.255637 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.255637 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.255744 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.256370 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:39.256381 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:39.258031 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:39.259749 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:39.259757 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:39.259760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:39.259998 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:39.260186 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:39.260186 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:39.260186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:39.260313 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:39.260591 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:39.260595 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:39.260721 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 05:01:39.260807 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 05:01:39.260942 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260808 05:01:39.261682 32361 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45123
I20260808 05:01:39.261798 32361 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:39.262003 32361 heartbeater.cc:511] Master 127.3.174.190:45123 requested a full tablet report, sending...
I20260808 05:01:39.262420 31759 ts_manager.cc:194] Registered new tserver with Master: c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
I20260808 05:01:39.262789 31759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:33111
W20260808 05:01:39.343511 32366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:39.343757 32366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:39.343793 32366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:39.345052 32366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:39.345104 32366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:39.345141 32366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:01:39.346764 32366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:41869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=36421
--tserver_master_addrs=127.3.174.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:39.347034 32366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:39.347247 32366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:39.352530 32373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:39.352591 32378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:39.352562 32375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:39.352584 32366 server_base.cc:1034] running on GCE node
I20260808 05:01:39.352908 32366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:39.353173 32366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:39.354310 32366 hybrid_clock.cc:648] HybridClock initialized: now 1786165299354297 us; error 29 us; skew 500 ppm
Aug 08 05:01:39 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165299, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:39.356985 32366 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:01:39.357520 32366 webserver.cc:460] Webserver started at http://127.3.174.131:36421/ using document root <none> and password file <none>
I20260808 05:01:39.357728 32366 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:39.357784 32366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:39.359148 32366 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:39.359745 32392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:39.359901 32366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:01:39.359987 32366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "01bb19fd75124364b9982a61157afe4b"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "bdb697f5f7067ba1c2f78f001b8ba82a"
server_key_iv: "8804c060236e14587473f0a4f52f8740"
server_key_version: "encryptionkey@0"
I20260808 05:01:39.360359 32366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:39.368981 32366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:39.371160 32366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:39.371294 32366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:39.371531 32366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:01:39.371850 32366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:01:39.371899 32366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:39.371933 32366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:01:39.371956 32366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:39.377848 32366 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:41869
I20260808 05:01:39.377908 32593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:41869 every 8 connection(s)
I20260808 05:01:39.378263 32366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:01:39.383944 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 32366
Aug 08 05:01:39 dist-test-slave-gjw7 krb5kdc[30699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165299, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:01:39.388064 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.379038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:60355 (local address 127.3.174.190:45123)
0808 05:01:39.379107 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.379110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.379338 (+ 228us) server_negotiation.cc:407] Connection header received
0808 05:01:39.379871 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.379873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.379883 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.379937 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:39.380928 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.381468 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.382512 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.382704 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.382759 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.385056 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:39.385057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:39.385057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:39.385064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:39.386378 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.386781 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.386782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.386783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.386840 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.387317 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.387318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.387318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.387442 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:39.387503 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.387855 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.387943 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 05:01:39.388530 32601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.379035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:60355)
0808 05:01:39.379212 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:39.379253 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:39.379777 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:39.380010 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:39.380013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:39.380635 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:01:39.380842 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.380847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.381549 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.381550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.382436 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.382439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.382767 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.382768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.382869 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.383340 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:01:39.383349 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:01:39.384972 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:01:39.386459 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:39.386471 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:39.386474 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:39.386710 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:39.386896 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:01:39.386897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:01:39.386897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:01:39.387079 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:01:39.387511 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:01:39.387521 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:01:39.387682 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 05:01:39.387809 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 05:01:39.387968 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260808 05:01:39.389143 32594 heartbeater.cc:347] Connected to a master server at 127.3.174.190:45123
I20260808 05:01:39.389262 32594 heartbeater.cc:464] Registering TS with master...
I20260808 05:01:39.389519 32594 heartbeater.cc:511] Master 127.3.174.190:45123 requested a full tablet report, sending...
I20260808 05:01:39.389953 31759 ts_manager.cc:194] Registered new tserver with Master: 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869)
I20260808 05:01:39.390407 31759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:60355
I20260808 05:01:39.390836 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.384429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.3.174.190:45123)
0808 05:01:39.384533 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.384535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.384621 (+ 86us) server_negotiation.cc:407] Connection header received
0808 05:01:39.384673 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.384674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.384685 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.384748 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:39.385350 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.385828 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.386596 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.386790 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.386845 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.388116 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:39.388117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:39.388117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:39.388125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:39.389693 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.390085 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.390085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.390086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.390148 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.390335 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.390335 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.390335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.390442 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:39.390621 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.390661 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.390745 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":55,"threads_started":1}
I20260808 05:01:39.391454 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:01:39.393988 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.391788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.3.174.190:45123)
0808 05:01:39.391830 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.391832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.391915 (+ 83us) server_negotiation.cc:407] Connection header received
0808 05:01:39.391965 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.391966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.391997 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.392057 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:01:39.392134 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.392594 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.393333 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.393547 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.393600 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.393709 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:01:39.393754 (+ 45us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 05:01:39.393943 (+ 189us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46172: token expired
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 05:01:39.394033 32605 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46172: token expired
W20260808 05:01:39.394095 31622 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.174.190:45123)
I20260808 05:01:39.399715 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.394358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46184 (local address 127.3.174.190:45123)
0808 05:01:39.394417 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.394419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.394448 (+ 29us) server_negotiation.cc:407] Connection header received
0808 05:01:39.394506 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.394507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.394515 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.394559 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:01:39.395001 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.395438 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.396117 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.396290 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.396342 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.397446 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:01:39.397447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:01:39.397447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:01:39.397454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:01:39.398681 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.398995 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.398996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.398996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.399046 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:01:39.399209 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:01:39.399210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:01:39.399210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:01:39.399355 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:01:39.399454 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.399547 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.399625 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 05:01:39.400893 31759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:01:39.402578 31759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:01:39.404976 31986 tablet_service.cc:1467] Processing CreateTablet for tablet 3853305796ad49b6be6461108902d921 (DEFAULT_TABLE table=test-table [id=81182492a8b94c7bbf4851735717a92f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:39.405205 31986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3853305796ad49b6be6461108902d921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:39.406945 31908 tablet_bootstrap.cc:492] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Bootstrap starting.
I20260808 05:01:39.407366 31908 tablet_bootstrap.cc:654] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:39.408142 31908 tablet_bootstrap.cc:492] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: No bootstrap required, opened a new log
I20260808 05:01:39.408201 31908 ts_tablet_manager.cc:1397] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 05:01:39.408353 31908 raft_consensus.cc:348] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:01:39.408443 31908 raft_consensus.cc:374] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:39.408502 31908 raft_consensus.cc:729] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14afd3da8e144fd792cec0d10c77f5e4, State: Initialized, Role: FOLLOWER
I20260808 05:01:39.408550 31908 consensus_queue.cc:260] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:01:39.408654 31908 ts_tablet_manager.cc:1428] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:39.408763 32609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.404540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:40343 (local address 127.0.0.1:59524)
0808 05:01:39.404744 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:39.404752 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:39.404794 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:39.405277 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:39.405278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:39.405280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:39.405365 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.405367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.406403 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.406404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.407546 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.407549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.408485 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.408487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.408556 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.408561 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:01:39.408630 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 05:01:39.408681 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":13,"thread_start_us":127,"threads_started":1}
I20260808 05:01:39.408779 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.404823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59524 (local address 127.3.174.130:40343)
0808 05:01:39.405086 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.405092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.405106 (+ 14us) server_negotiation.cc:407] Connection header received
0808 05:01:39.405148 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.405151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.405168 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.405466 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:39.405565 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.406307 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.407643 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.408403 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.408480 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.408557 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.408650 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.408697 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":111,"threads_started":1}
I20260808 05:01:39.409639 32218 tablet_service.cc:1467] Processing CreateTablet for tablet 3853305796ad49b6be6461108902d921 (DEFAULT_TABLE table=test-table [id=81182492a8b94c7bbf4851735717a92f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:39.410056 32218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3853305796ad49b6be6461108902d921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:39.410252 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.404530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:41869 (local address 127.0.0.1:59762)
0808 05:01:39.404569 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:39.404581 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:39.404620 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:39.405275 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:39.405277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:39.405279 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:39.405527 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.405529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.406709 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.406710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.407849 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.407853 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.410021 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.410022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.410071 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.410075 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:01:39.410129 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 05:01:39.410181 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 05:01:39.410267 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.404647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.3.174.131:41869)
0808 05:01:39.405065 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.405071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.405082 (+ 11us) server_negotiation.cc:407] Connection header received
0808 05:01:39.405133 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.405135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.405153 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.405272 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:39.405594 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.406617 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.408754 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.409949 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.410022 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.410085 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.410145 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.410187 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":121,"threads_started":1}
I20260808 05:01:39.410943 32474 tablet_service.cc:1467] Processing CreateTablet for tablet 3853305796ad49b6be6461108902d921 (DEFAULT_TABLE table=test-table [id=81182492a8b94c7bbf4851735717a92f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:01:39.411263 32474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3853305796ad49b6be6461108902d921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:01:39.413089 32614 tablet_bootstrap.cc:492] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: Bootstrap starting.
I20260808 05:01:39.413932 32614 tablet_bootstrap.cc:654] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:39.414497 32616 tablet_bootstrap.cc:492] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: Bootstrap starting.
I20260808 05:01:39.414570 32614 log.cc:824] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:39.415285 32616 tablet_bootstrap.cc:654] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: Neither blocks nor log segments found. Creating new log.
I20260808 05:01:39.415387 32614 tablet_bootstrap.cc:492] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: No bootstrap required, opened a new log
I20260808 05:01:39.415474 32614 ts_tablet_manager.cc:1397] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:01:39.415753 32616 log.cc:824] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:39.416414 32616 tablet_bootstrap.cc:492] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: No bootstrap required, opened a new log
I20260808 05:01:39.416476 32616 ts_tablet_manager.cc:1397] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:01:39.416922 32614 raft_consensus.cc:348] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:01:39.417070 32614 raft_consensus.cc:374] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:39.417092 32614 raft_consensus.cc:729] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c897bcef3ca24482b9e2d4e4c99ac99c, State: Initialized, Role: FOLLOWER
I20260808 05:01:39.417202 32614 consensus_queue.cc:260] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:01:39.417529 32361 heartbeater.cc:503] Master 127.3.174.190:45123 was elected leader, sending a full tablet report...
I20260808 05:01:39.417544 32614 ts_tablet_manager.cc:1428] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:01:39.418077 32616 raft_consensus.cc:348] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:01:39.418337 32616 raft_consensus.cc:374] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:01:39.418385 32616 raft_consensus.cc:729] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bb19fd75124364b9982a61157afe4b, State: Initialized, Role: FOLLOWER
I20260808 05:01:39.418582 32616 consensus_queue.cc:260] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:01:39.418897 32616 ts_tablet_manager.cc:1428] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:01:39.418907 32594 heartbeater.cc:503] Master 127.3.174.190:45123 was elected leader, sending a full tablet report...
W20260808 05:01:39.501634 32362 tablet.cc:2369] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 05:01:39.618209 32101 tablet.cc:2369] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 05:01:39.628891 32600 tablet.cc:2369] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:01:39.668486 32110 raft_consensus.cc:482] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:01:39.668658 32110 raft_consensus.cc:504] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:01:39.669056 32110 leader_election.cc:288] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869), c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
I20260808 05:01:39.671888 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.669434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:42661 (local address 127.3.174.131:41869)
0808 05:01:39.669479 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.669481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.669490 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:01:39.669552 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.669553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.669562 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.669616 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:39.669754 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.670236 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.671202 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.671576 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.671639 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.671649 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.671776 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.671816 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 05:01:39.671914 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.669397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:55531 (local address 127.3.174.130:40343)
0808 05:01:39.669449 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.669451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.669484 (+ 33us) server_negotiation.cc:407] Connection header received
0808 05:01:39.669566 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.669567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.669575 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.669620 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:01:39.669777 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.670269 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.671211 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.671591 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.671654 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.671663 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.671801 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.671841 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 05:01:39.671932 32625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.669251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:40343 (local address 127.3.174.129:55531)
0808 05:01:39.669418 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:39.669431 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:39.669496 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:39.669629 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:39.669630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:39.669632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:39.669708 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.669709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.670341 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.670342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.671114 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.671116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.671663 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.671665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.671737 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.671740 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:01:39.671786 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 05:01:39.671831 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260808 05:01:39.671943 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.669253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:41869 (local address 127.3.174.129:42661)
0808 05:01:39.669419 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:39.669431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 05:01:39.669494 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:01:39.669623 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:01:39.669624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:01:39.669626 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:01:39.669695 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.669697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.670308 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.670309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.671104 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:01:39.671106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.671649 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.671651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:01:39.671715 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.671719 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:01:39.671763 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 05:01:39.671810 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260808 05:01:39.672407 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3853305796ad49b6be6461108902d921" candidate_uuid: "14afd3da8e144fd792cec0d10c77f5e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bb19fd75124364b9982a61157afe4b" is_pre_election: true
I20260808 05:01:39.672458 32259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3853305796ad49b6be6461108902d921" candidate_uuid: "14afd3da8e144fd792cec0d10c77f5e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" is_pre_election: true
I20260808 05:01:39.672608 32259 raft_consensus.cc:2454] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14afd3da8e144fd792cec0d10c77f5e4 in term 0.
I20260808 05:01:39.672609 32512 raft_consensus.cc:2454] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14afd3da8e144fd792cec0d10c77f5e4 in term 0.
I20260808 05:01:39.672866 31893 leader_election.cc:302] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bb19fd75124364b9982a61157afe4b, 14afd3da8e144fd792cec0d10c77f5e4; no voters:
I20260808 05:01:39.673018 32110 raft_consensus.cc:2781] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:01:39.673085 32110 raft_consensus.cc:482] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:01:39.673102 32110 raft_consensus.cc:3037] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:39.673890 32110 raft_consensus.cc:504] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:01:39.674029 32110 leader_election.cc:288] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 1 election: Requested vote from peers 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869), c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
I20260808 05:01:39.674211 32512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3853305796ad49b6be6461108902d921" candidate_uuid: "14afd3da8e144fd792cec0d10c77f5e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bb19fd75124364b9982a61157afe4b"
I20260808 05:01:39.674260 32259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3853305796ad49b6be6461108902d921" candidate_uuid: "14afd3da8e144fd792cec0d10c77f5e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c"
I20260808 05:01:39.674330 32259 raft_consensus.cc:3037] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:39.674307 32512 raft_consensus.cc:3037] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:01:39.675258 32512 raft_consensus.cc:2454] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14afd3da8e144fd792cec0d10c77f5e4 in term 1.
I20260808 05:01:39.675258 32259 raft_consensus.cc:2454] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14afd3da8e144fd792cec0d10c77f5e4 in term 1.
I20260808 05:01:39.675422 31893 leader_election.cc:302] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bb19fd75124364b9982a61157afe4b, 14afd3da8e144fd792cec0d10c77f5e4; no voters:
I20260808 05:01:39.675536 32110 raft_consensus.cc:2781] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:01:39.675666 32110 raft_consensus.cc:686] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 LEADER]: Becoming Leader. State: Replica: 14afd3da8e144fd792cec0d10c77f5e4, State: Running, Role: LEADER
I20260808 05:01:39.675736 32110 consensus_queue.cc:237] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:01:39.676560 31757 catalog_manager.cc:5495] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14afd3da8e144fd792cec0d10c77f5e4 (127.3.174.129). New cstate: current_term: 1 leader_uuid: "14afd3da8e144fd792cec0d10c77f5e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } health_report { overall_health: HEALTHY } } }
I20260808 05:01:39.694687 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.691662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.3.174.129:39783)
0808 05:01:39.691823 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 05:01:39.691825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:01:39.691830 (+ 5us) server_negotiation.cc:407] Connection header received
0808 05:01:39.691873 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:01:39.691873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:01:39.691894 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:01:39.691966 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:01:39.692088 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.692613 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.693344 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:01:39.694086 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:01:39.694152 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:01:39.694336 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:01:39.694452 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:01:39.694516 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 05:01:39.694554 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 05:01:39.694578 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260808 05:01:39.696435 32259 raft_consensus.cc:1261] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [term 1 FOLLOWER]: Refusing update from remote peer 14afd3da8e144fd792cec0d10c77f5e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:01:39.696435 32512 raft_consensus.cc:1261] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [term 1 FOLLOWER]: Refusing update from remote peer 14afd3da8e144fd792cec0d10c77f5e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:01:39.696818 32629 consensus_queue.cc:1035] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:01:39.696974 32110 consensus_queue.cc:1035] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:01:39.699524 32639 mvcc.cc:205] Tried to move back new op lower bound from 7316133067553689600 to 7316133067472547840. Current Snapshot: MvccSnapshot[applied={T|T < 7316133067553689600}]
I20260808 05:01:39.700276 32646 mvcc.cc:205] Tried to move back new op lower bound from 7316133067553689600 to 7316133067472547840. Current Snapshot: MvccSnapshot[applied={T|T < 7316133067553689600}]
I20260808 05:01:39.701946 31757 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:01:39.702044 31757 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:01:39.702759 31757 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Sending DeleteTablet for 3 replicas of tablet 3853305796ad49b6be6461108902d921
I20260808 05:01:39.703037 31986 tablet_service.cc:1514] Processing DeleteTablet for tablet 3853305796ad49b6be6461108902d921 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:39 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:47980
I20260808 05:01:39.703136 32474 tablet_service.cc:1514] Processing DeleteTablet for tablet 3853305796ad49b6be6461108902d921 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:39 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:59762
I20260808 05:01:39.703188 32218 tablet_service.cc:1514] Processing DeleteTablet for tablet 3853305796ad49b6be6461108902d921 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:01:39 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:59524
I20260808 05:01:39.703261 32659 tablet_replica.cc:330] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: stopping tablet replica
I20260808 05:01:39.703310 32659 raft_consensus.cc:2229] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:01:39.703403 32659 raft_consensus.cc:2258] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:39.703423 32661 tablet_replica.cc:330] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: stopping tablet replica
I20260808 05:01:39.703540 32661 raft_consensus.cc:2229] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:01:39.703670 32661 raft_consensus.cc:2258] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:39.703698 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 31826
I20260808 05:01:39.703738 32659 ts_tablet_manager.cc:1905] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:39.709164 32661 ts_tablet_manager.cc:1905] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:39.713261 32661 ts_tablet_manager.cc:1918] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:39.713397 32661 log.cc:1196] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/3853305796ad49b6be6461108902d921
I20260808 05:01:39.713670 32661 ts_tablet_manager.cc:1939] T 3853305796ad49b6be6461108902d921 P c897bcef3ca24482b9e2d4e4c99ac99c: Deleting consensus metadata
I20260808 05:01:39.714159 31735 catalog_manager.cc:4841] TS c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343): tablet 3853305796ad49b6be6461108902d921 (table test-table [id=81182492a8b94c7bbf4851735717a92f]) successfully deleted
I20260808 05:01:39.715638 32660 tablet_replica.cc:330] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: stopping tablet replica
I20260808 05:01:39.715894 32660 raft_consensus.cc:2229] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:01:39.716136 32660 raft_consensus.cc:2258] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:01:39.716720 32660 ts_tablet_manager.cc:1905] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:01:39.718096 32660 ts_tablet_manager.cc:1918] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:01:39.718163 32660 log.cc:1196] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/3853305796ad49b6be6461108902d921
I20260808 05:01:39.718415 32660 ts_tablet_manager.cc:1939] T 3853305796ad49b6be6461108902d921 P 01bb19fd75124364b9982a61157afe4b: Deleting consensus metadata
I20260808 05:01:39.718918 31732 catalog_manager.cc:4841] TS 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869): tablet 3853305796ad49b6be6461108902d921 (table test-table [id=81182492a8b94c7bbf4851735717a92f]) successfully deleted
W20260808 05:01:39.720803 31731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:39783 (error 108)
I20260808 05:01:39.721205 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 32109
I20260808 05:01:39.721432 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:01:39.721206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39783 (local address 127.0.0.1:42776)
0808 05:01:39.721337 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 05:01:39.721395 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:39783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 05:01:39.738026 31731 catalog_manager.cc:4584] TS 14afd3da8e144fd792cec0d10c77f5e4 (127.3.174.129:39783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3853305796ad49b6be6461108902d921: Network error: Client connection negotiation failed: client connection to 127.3.174.129:39783: connect: Connection refused (error 111)
I20260808 05:01:39.748646 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 32366
I20260808 05:01:39.767952 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 31699
I20260808 05:01:59.783497 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45123
--webserver_interface=127.3.174.190
--webserver_port=43933
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:45123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:01:59.869506 32664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:01:59.869709 32664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:01:59.869730 32664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:01:59.871157 32664 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 05:01:59.871255 32664 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 05:01:59.871268 32664 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 05:01:59.871279 32664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:01:59.871296 32664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:01:59.871308 32664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:01:59.871320 32664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:01:59.871352 32664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:01:59.873037 32664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:45123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:45123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=43933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:01:59.873322 32664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:01:59.873559 32664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:01:59.878532 32672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:01:59.878707 32676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:59.878708 32664 server_base.cc:1034] running on GCE node
W20260808 05:01:59.878551 32673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:01:59.879087 32664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:01:59.879428 32664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:01:59.880596 32664 hybrid_clock.cc:648] HybridClock initialized: now 1786165319880573 us; error 40 us; skew 500 ppm
Aug 08 05:01:59 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165319, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:01:59.883698 32664 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:01:59.884244 32664 webserver.cc:460] Webserver started at http://127.3.174.190:43933/ using document root <none> and password file <none>
I20260808 05:01:59.884527 32664 fs_manager.cc:359] Metadata directory not provided
I20260808 05:01:59.884594 32664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:01:59.886020 32664 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:01:59.886670 32686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:01:59.886822 32664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:01:59.886909 32664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "b2c53df135dd4d10b95f69d28ca24863"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "5a3b04675a5254c8d864f62c93da4aaf"
server_key_iv: "ee96dec75be68fd86369aead0a321cb1"
server_key_version: "encryptionkey@0"
I20260808 05:01:59.887264 32664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:01:59.895057 32664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:01:59.897048 32664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:01:59.897176 32664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:01:59.901209 32664 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:45123
I20260808 05:01:59.901240 312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:45123 every 8 connection(s)
I20260808 05:01:59.901652 32664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:01:59.903131 313 sys_catalog.cc:263] Verifying existing consensus state
I20260808 05:01:59.903761 313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Bootstrap starting.
I20260808 05:01:59.909842 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 32664
I20260808 05:01:59.910107 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:39783
--local_ip_for_outbound_sockets=127.3.174.129
--tserver_master_addrs=127.3.174.190:45123
--webserver_port=36041
--webserver_interface=127.3.174.129
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:01:59.912498 313 log.cc:824] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Log is configured to *not* fsync() on all Append() calls
I20260808 05:01:59.914777 313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 05:01:59.915059 313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Bootstrap complete.
I20260808 05:01:59.917209 313 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:59.917486 313 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c53df135dd4d10b95f69d28ca24863, State: Initialized, Role: FOLLOWER
I20260808 05:01:59.917683 313 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:59.917773 313 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:01:59.917822 313 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:01:59.917860 313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 2 FOLLOWER]: Advancing to term 3
I20260808 05:01:59.919168 313 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:59.919267 313 leader_election.cc:302] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2c53df135dd4d10b95f69d28ca24863; no voters:
I20260808 05:01:59.919431 313 leader_election.cc:288] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 05:01:59.919480 319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 05:01:59.919616 319 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [term 3 LEADER]: Becoming Leader. State: Replica: b2c53df135dd4d10b95f69d28ca24863, State: Running, Role: LEADER
I20260808 05:01:59.919726 313 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:01:59.919708 319 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } }
I20260808 05:01:59.920047 319 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2c53df135dd4d10b95f69d28ca24863. Latest consensus state: current_term: 3 leader_uuid: "b2c53df135dd4d10b95f69d28ca24863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } } }
I20260808 05:01:59.920136 319 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:59.920553 326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:01:59.920573 323 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b2c53df135dd4d10b95f69d28ca24863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c53df135dd4d10b95f69d28ca24863" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 45123 } } }
I20260808 05:01:59.920640 323 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863 [sys.catalog]: This master's current role is: LEADER
I20260808 05:01:59.921434 326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:01:59.921548 326 catalog_manager.cc:1259] Loaded cluster ID: 36919a836f2e438e81a988f9b612bf14
I20260808 05:01:59.921579 326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:01:59.922727 326 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:01:59.922915 326 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Loaded TSK: 0
I20260808 05:01:59.922967 326 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2c53df135dd4d10b95f69d28ca24863: Loaded TSK: 1
F20260808 05:01:59.923195 326 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb253faa9ad google::LogMessage::Fail() at ??:0
@ 0x7fb253fad88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb253faa4de google::LogMessage::Flush() at ??:0
@ 0x7fb253faba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb256717104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb256714d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb25bf041cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb25bf0448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb25bf04594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fb25becac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb25bed4ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb255e780d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb255e6d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb256cd16db start_thread at ??:0
@ 0x7fb252aa861f clone at ??:0
W20260808 05:02:00.027055 315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:00.027343 315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:00.027381 315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:00.029021 315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:00.029080 315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:00.029120 315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:02:00.031239 315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:39783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=36041
--tserver_master_addrs=127.3.174.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:00.031513 315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:00.031780 315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:00.037091 378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:00.037114 379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:00.037127 383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:00.037493 315 server_base.cc:1034] running on GCE node
I20260808 05:02:00.037707 315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:00.037979 315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:00.039178 315 hybrid_clock.cc:648] HybridClock initialized: now 1786165320039157 us; error 34 us; skew 500 ppm
Aug 08 05:02:00 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165320, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:00.042114 315 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:02:00.042538 315 webserver.cc:460] Webserver started at http://127.3.174.129:36041/ using document root <none> and password file <none>
I20260808 05:02:00.042759 315 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:00.042804 315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:00.044094 315 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:02:00.044875 394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:00.045065 315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:02:00.045156 315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "14afd3da8e144fd792cec0d10c77f5e4"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "b38fd9046c25234a76bd77c30b86cd7d"
server_key_iv: "14daae5fbe2025eb0c936efc7e1094c0"
server_key_version: "encryptionkey@0"
I20260808 05:02:00.045536 315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:00.069798 315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:00.072407 315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:00.072558 315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:00.072801 315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:00.073280 407 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 05:02:00.075335 315 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 05:02:00.075385 315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 05:02:00.075428 315 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 05:02:00.076119 315 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 05:02:00.076164 315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 05:02:00.076251 407 tablet_bootstrap.cc:492] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Bootstrap starting.
I20260808 05:02:00.078392 407 log.cc:824] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:00.081600 407 tablet_bootstrap.cc:492] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 05:02:00.082003 407 tablet_bootstrap.cc:492] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Bootstrap complete.
I20260808 05:02:00.082166 407 ts_tablet_manager.cc:1397] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260808 05:02:00.083247 315 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:39783
I20260808 05:02:00.083443 616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:39783 every 8 connection(s)
I20260808 05:02:00.083626 315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:02:00.084127 407 raft_consensus.cc:348] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:02:00.084389 407 raft_consensus.cc:729] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14afd3da8e144fd792cec0d10c77f5e4, State: Initialized, Role: FOLLOWER
I20260808 05:02:00.084530 407 consensus_queue.cc:260] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:02:00.084801 407 ts_tablet_manager.cc:1428] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 05:02:00.086422 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 315
I20260808 05:02:00.086669 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:40343
--local_ip_for_outbound_sockets=127.3.174.130
--tserver_master_addrs=127.3.174.190:45123
--webserver_port=33557
--webserver_interface=127.3.174.130
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:00.149387 622 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:00.084345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:60401)
0808 05:02:00.084530 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:00.084573 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:00.084734 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:00.149145 (+ 64411us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
W20260808 05:02:00.149622 617 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:45123 (0 consecutive failures): Network error: Failed to ping master at 127.3.174.190:45123: Client connection negotiation failed: client connection to 127.3.174.190:45123: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 05:02:00.149982 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:00.149839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:45225)
0808 05:02:00.149894 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:00.149947 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 05:02:00.150285 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:00.150208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:38225)
0808 05:02:00.150243 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:00.150267 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 05:02:00.150493 617 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 05:02:00.174196 625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:00.174408 625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:00.174431 625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:00.175626 625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:00.175666 625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:00.175691 625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:02:00.177650 625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:40343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=33557
--tserver_master_addrs=127.3.174.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:00.177891 625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:00.178114 625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:00.182960 638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:00.183005 645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:00.182991 636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:00.183192 625 server_base.cc:1034] running on GCE node
I20260808 05:02:00.183382 625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:00.183598 625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:00.184749 625 hybrid_clock.cc:648] HybridClock initialized: now 1786165320184740 us; error 27 us; skew 500 ppm
Aug 08 05:02:00 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165320, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:00.187228 625 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:02:00.187654 625 webserver.cc:460] Webserver started at http://127.3.174.130:33557/ using document root <none> and password file <none>
I20260808 05:02:00.187886 625 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:00.187937 625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:00.189221 625 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:02:00.189780 666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:00.189939 625 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:00.190024 625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "c897bcef3ca24482b9e2d4e4c99ac99c"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "c6ddc990b9eb317fbacab52e2292439d"
server_key_iv: "7a9030f6368a89f5bd387d90f65fd4e6"
server_key_version: "encryptionkey@0"
I20260808 05:02:00.190351 625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:00.211661 625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:00.213927 625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:00.214131 625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:00.214365 625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:00.214675 625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:00.214711 625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:00.214742 625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:00.214764 625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:00.220789 625 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:40343
I20260808 05:02:00.220830 863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:40343 every 8 connection(s)
I20260808 05:02:00.221159 625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 05:02:00.222185 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:00.221813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:51171)
0808 05:02:00.221967 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:00.222039 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
W20260808 05:02:00.222601 864 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:45123 (0 consecutive failures): Network error: Failed to ping master at 127.3.174.190:45123: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
I20260808 05:02:00.222862 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:00.222775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:42079)
0808 05:02:00.222814 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:00.222840 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 05:02:00.223117 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:00.223045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:39997)
0808 05:02:00.223077 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:00.223097 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 05:02:00.223209 864 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 05:02:00.225929 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 625
I20260808 05:02:00.226186 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:41869
--local_ip_for_outbound_sockets=127.3.174.131
--tserver_master_addrs=127.3.174.190:45123
--webserver_port=36421
--webserver_interface=127.3.174.131
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:00.309250 867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:00.309485 867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:00.309516 867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:00.310638 867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:00.310678 867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:00.310711 867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:02:00.312108 867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:41869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=36421
--tserver_master_addrs=127.3.174.190:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:00.312335 867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:00.312539 867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:00.317426 884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:00.317508 879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:00.317459 881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:00.317616 867 server_base.cc:1034] running on GCE node
I20260808 05:02:00.317799 867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:00.318017 867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:00.319165 867 hybrid_clock.cc:648] HybridClock initialized: now 1786165320319137 us; error 39 us; skew 500 ppm
Aug 08 05:02:00 dist-test-slave-gjw7 krb5kdc[30699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165320, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:00.321631 867 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:02:00.322036 867 webserver.cc:460] Webserver started at http://127.3.174.131:36421/ using document root <none> and password file <none>
I20260808 05:02:00.322230 867 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:00.322275 867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:00.323601 867 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:00.324123 894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:00.324278 867 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:00.324365 867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "01bb19fd75124364b9982a61157afe4b"
format_stamp: "Formatted at 2026-08-08 05:01:18 on dist-test-slave-gjw7"
server_key: "bdb697f5f7067ba1c2f78f001b8ba82a"
server_key_iv: "8804c060236e14587473f0a4f52f8740"
server_key_version: "encryptionkey@0"
I20260808 05:02:00.324697 867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:00.351818 867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:00.353667 867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:00.353780 867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:00.353979 867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:00.354251 867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:00.354282 867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:00.354305 867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:00.354316 867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:00.359886 867 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:41869
I20260808 05:02:00.359963 1102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:41869 every 8 connection(s)
I20260808 05:02:00.360275 867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:02:00.361284 1109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:00.360890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:57491)
0808 05:02:00.361053 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:00.361126 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
W20260808 05:02:00.361694 1105 heartbeater.cc:644] Failed to heartbeat to 127.3.174.190:45123 (0 consecutive failures): Network error: Failed to ping master at 127.3.174.190:45123: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
I20260808 05:02:00.361944 1109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:00.361860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:58253)
0808 05:02:00.361900 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:00.361925 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 05:02:00.362072 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 867
I20260808 05:02:00.362260 1109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:00.362194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:58051)
0808 05:02:00.362222 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:00.362244 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 05:02:00.362448 1105 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 05:02:01.151038 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:01.150808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:55187)
0808 05:02:01.150937 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:01.150974 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260808 05:02:01.223723 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:01.223521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:46695)
0808 05:02:01.223634 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:01.223669 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260808 05:02:01.326633 1118 raft_consensus.cc:482] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:01.326767 1118 raft_consensus.cc:504] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:02:01.327124 1118 leader_election.cc:288] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869), c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
Aug 08 05:02:01 dist-test-slave-gjw7 krb5kdc[30699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165320, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.130@KRBTEST.COM
Aug 08 05:02:01 dist-test-slave-gjw7 krb5kdc[30699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165320, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.131@KRBTEST.COM
I20260808 05:02:01.338982 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:01.327461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:46057 (local address 127.3.174.130:40343)
0808 05:02:01.327602 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:01.327610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:01.327640 (+ 30us) server_negotiation.cc:407] Connection header received
0808 05:02:01.328172 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:01.328174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:01.328235 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:01.328383 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:01.329396 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:01.330843 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:01.332353 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:01.332984 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:01.333095 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:01.335557 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:01.335561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:01.335562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:01.335575 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:01.337428 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:01.337852 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:01.337855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:01.337856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:01.337949 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:01.338167 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:01.338167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:01.338168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:01.338402 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:01.338473 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:01.338734 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 05:02:01.338866 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260808 05:02:01.338981 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:01.327403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:58103 (local address 127.3.174.131:41869)
0808 05:02:01.327556 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:01.327568 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:01.327593 (+ 25us) server_negotiation.cc:407] Connection header received
0808 05:02:01.328169 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:01.328172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:01.328236 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:01.328383 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:01.329445 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:01.330845 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:01.332368 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:01.332983 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:01.333093 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:01.335864 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:01.335869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:01.335870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:01.335882 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:01.337535 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:01.337877 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:01.337880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:01.337880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:01.337971 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:01.338165 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:01.338166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:01.338166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:01.338402 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:01.338473 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:01.338735 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 05:02:01.338870 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":54,"threads_started":1}
I20260808 05:02:01.338971 1119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:01.327273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:40343 (local address 127.3.174.129:46057)
0808 05:02:01.327430 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:01.327440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:01.327493 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:01.328463 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:01.328467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:01.328920 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:01.329242 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:01.329251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:01.330953 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:01.330954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:01.332261 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:01.332265 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:01.333081 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:01.333081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:01.333251 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:01.333859 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:01.333869 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:01.335450 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:01.337503 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:01.337510 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:01.337512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:01.337766 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:01.338009 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:01.338010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:01.338010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:01.338108 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:01.338474 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:01.338475 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:01.338586 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 05:02:01.338673 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 05:02:01.338861 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":78,"thread_start_us":64,"threads_started":1}
I20260808 05:02:01.339066 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:01.327272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:41869 (local address 127.3.174.129:58103)
0808 05:02:01.327321 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:01.327333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:01.327397 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:01.328463 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:01.328467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:01.329260 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:01.329346 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:01.329349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:01.330953 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:01.330954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:01.332261 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:01.332265 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:01.333079 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:01.333080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:01.333251 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:01.333996 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:01.334005 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:01.335767 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:01.337606 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:01.337607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:01.337608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:01.337807 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:01.338029 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:01.338029 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:01.338030 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:01.338110 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:01.338465 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:01.338472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:01.338586 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 05:02:01.338673 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 05:02:01.338935 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":177}
W20260808 05:02:01.339970 402 leader_election.cc:341] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343): Not found: Tablet not found: 3853305796ad49b6be6461108902d921
W20260808 05:02:01.340068 398 leader_election.cc:341] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869): Not found: Tablet not found: 3853305796ad49b6be6461108902d921
I20260808 05:02:01.340130 398 leader_election.cc:302] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14afd3da8e144fd792cec0d10c77f5e4; no voters: 01bb19fd75124364b9982a61157afe4b, c897bcef3ca24482b9e2d4e4c99ac99c
I20260808 05:02:01.340232 1118 raft_consensus.cc:2726] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 05:02:01.362996 1123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:01.362773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:40109)
0808 05:02:01.362903 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:01.362962 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260808 05:02:02.151691 1126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:02.151464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:45659)
0808 05:02:02.151601 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:02.151642 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260808 05:02:02.224340 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:02.224138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:49771)
0808 05:02:02.224266 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:02.224302 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260808 05:02:02.363636 1129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:02.363385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:48899)
0808 05:02:02.363528 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:02.363569 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260808 05:02:03.055333 1130 raft_consensus.cc:482] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:03.055434 1130 raft_consensus.cc:504] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:02:03.055581 1130 leader_election.cc:288] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869), c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
W20260808 05:02:03.055996 402 leader_election.cc:341] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343): Not found: Tablet not found: 3853305796ad49b6be6461108902d921
W20260808 05:02:03.056087 398 leader_election.cc:341] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869): Not found: Tablet not found: 3853305796ad49b6be6461108902d921
I20260808 05:02:03.056139 398 leader_election.cc:302] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14afd3da8e144fd792cec0d10c77f5e4; no voters: 01bb19fd75124364b9982a61157afe4b, c897bcef3ca24482b9e2d4e4c99ac99c
I20260808 05:02:03.056211 1130 raft_consensus.cc:2726] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 05:02:03.152354 1133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:03.152118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:50921)
0808 05:02:03.152245 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:03.152291 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260808 05:02:03.225003 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:03.224785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:42081)
0808 05:02:03.224919 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:03.224953 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260808 05:02:03.364272 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:03.364054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:58549)
0808 05:02:03.364191 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:03.364227 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260808 05:02:04.153115 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:04.152825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:56543)
0808 05:02:04.153006 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:04.153053 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260808 05:02:04.225646 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:04.225439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:36465)
0808 05:02:04.225569 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:04.225604 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260808 05:02:04.364871 1149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:04.364679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:54927)
0808 05:02:04.364795 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:04.364829 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260808 05:02:05.112895 1150 raft_consensus.cc:482] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:05.113022 1150 raft_consensus.cc:504] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:02:05.113340 1150 leader_election.cc:288] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869), c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
W20260808 05:02:05.113551 402 leader_election.cc:341] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343): Not found: Tablet not found: 3853305796ad49b6be6461108902d921
W20260808 05:02:05.113616 398 leader_election.cc:341] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869): Not found: Tablet not found: 3853305796ad49b6be6461108902d921
I20260808 05:02:05.113687 398 leader_election.cc:302] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14afd3da8e144fd792cec0d10c77f5e4; no voters: 01bb19fd75124364b9982a61157afe4b, c897bcef3ca24482b9e2d4e4c99ac99c
I20260808 05:02:05.113772 1150 raft_consensus.cc:2726] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 05:02:05.153796 1155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:05.153574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:35749)
0808 05:02:05.153701 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:05.153742 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260808 05:02:05.226253 1156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:05.226050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:57513)
0808 05:02:05.226173 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:05.226210 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260808 05:02:05.365468 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:05.365236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:39947)
0808 05:02:05.365388 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:05.365425 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260808 05:02:06.154520 1161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:06.154261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:45529)
0808 05:02:06.154419 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:06.154467 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":72,"threads_started":1}
I20260808 05:02:06.227026 1162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:06.226763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:50619)
0808 05:02:06.226929 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:06.226971 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260808 05:02:06.366091 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:06.365873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:47043)
0808 05:02:06.366006 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:06.366042 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260808 05:02:07.155174 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:07.154961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:32987)
0808 05:02:07.155085 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:07.155124 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260808 05:02:07.227713 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:07.227497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:34025)
0808 05:02:07.227635 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:07.227668 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260808 05:02:07.366807 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:07.366594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:34079)
0808 05:02:07.366716 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:07.366757 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260808 05:02:08.156041 1171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:08.155730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:58341)
0808 05:02:08.155937 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:08.155988 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":88,"threads_started":1}
I20260808 05:02:08.228463 1172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:08.228210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:52317)
0808 05:02:08.228362 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:08.228413 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260808 05:02:08.367475 1173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:08.367215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:52027)
0808 05:02:08.367379 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:08.367425 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
I20260808 05:02:09.156910 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:09.156616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:42469)
0808 05:02:09.156804 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:09.156857 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260808 05:02:09.229171 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:09.228928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:60383)
0808 05:02:09.229071 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:09.229122 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260808 05:02:09.368160 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:09.367933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:39063)
0808 05:02:09.368071 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:09.368113 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260808 05:02:09.969151 1181 raft_consensus.cc:482] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:09.969261 1181 raft_consensus.cc:504] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bb19fd75124364b9982a61157afe4b" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 41869 } } peers { permanent_uuid: "c897bcef3ca24482b9e2d4e4c99ac99c" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 40343 } } peers { permanent_uuid: "14afd3da8e144fd792cec0d10c77f5e4" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39783 } }
I20260808 05:02:09.969457 1181 leader_election.cc:288] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869), c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343)
W20260808 05:02:09.969875 398 leader_election.cc:341] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 01bb19fd75124364b9982a61157afe4b (127.3.174.131:41869): Not found: Tablet not found: 3853305796ad49b6be6461108902d921
W20260808 05:02:09.969997 402 leader_election.cc:341] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c897bcef3ca24482b9e2d4e4c99ac99c (127.3.174.130:40343): Not found: Tablet not found: 3853305796ad49b6be6461108902d921
I20260808 05:02:09.970060 402 leader_election.cc:302] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14afd3da8e144fd792cec0d10c77f5e4; no voters: 01bb19fd75124364b9982a61157afe4b, c897bcef3ca24482b9e2d4e4c99ac99c
I20260808 05:02:09.970127 1181 raft_consensus.cc:2726] T 3853305796ad49b6be6461108902d921 P 14afd3da8e144fd792cec0d10c77f5e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 05:02:10.157759 1182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:10.157519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:40863)
0808 05:02:10.157656 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:10.157706 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260808 05:02:10.229945 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:10.229692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:36461)
0808 05:02:10.229850 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:10.229895 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260808 05:02:10.368791 1189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:10.368573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:55987)
0808 05:02:10.368689 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:10.368727 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260808 05:02:11.158422 1190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:11.158192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:60193)
0808 05:02:11.158334 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:11.158372 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260808 05:02:11.230660 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:11.230453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:57913)
0808 05:02:11.230574 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:11.230611 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260808 05:02:11.369451 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:11.369214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:59143)
0808 05:02:11.369361 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:11.369402 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260808 05:02:12.159086 1199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:12.158853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:42991)
0808 05:02:12.158995 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:12.159036 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260808 05:02:12.231380 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:12.231150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:36985)
0808 05:02:12.231284 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:12.231329 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260808 05:02:12.370126 1203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:12.369924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:35907)
0808 05:02:12.370045 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:12.370077 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260808 05:02:13.159780 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:13.159546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:58175)
0808 05:02:13.159690 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:13.159730 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260808 05:02:13.232113 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:13.231882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:44559)
0808 05:02:13.232025 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:13.232066 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260808 05:02:13.370841 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:13.370595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:38459)
0808 05:02:13.370745 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:13.370789 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260808 05:02:14.160487 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:14.160253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:48879)
0808 05:02:14.160397 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:14.160436 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260808 05:02:14.232789 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:14.232569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:52705)
0808 05:02:14.232697 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:14.232737 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260808 05:02:14.371592 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:14.371361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:46269)
0808 05:02:14.371494 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:14.371538 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260808 05:02:15.161211 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:15.161000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.129:47141)
0808 05:02:15.161119 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:15.161155 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260808 05:02:15.233443 1217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:15.233235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.130:37391)
0808 05:02:15.233365 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:15.233396 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260808 05:02:15.365505 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 315
I20260808 05:02:15.372879 1219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:15.372668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:45123 (local address 127.3.174.131:39315)
0808 05:02:15.372788 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:15.372822 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.190:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":52,"threads_started":1}
I20260808 05:02:15.378261 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 625
I20260808 05:02:15.393054 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 867
2026-08-08T05:02:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58681 ms)
[----------] 30 tests from SecurityITest (238927 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:15 dist-test-slave-gjw7 krb5kdc[1224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:15 dist-test-slave-gjw7 krb5kdc[1224](info): set up 2 sockets
Aug 08 05:02:15 dist-test-slave-gjw7 krb5kdc[1224](info): commencing operation
krb5kdc: starting...
W20260808 05:02:16.596648 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:16 dist-test-slave-gjw7 krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:16.714138 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:36683
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:16.796901 1244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:16.797112 1244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:16.797142 1244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:16.798311 1244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:16.798362 1244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:16.798380 1244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:16.798398 1244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:16.798413 1244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:16.799876 1244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:36683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:36683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:16.800119 1244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:16.800323 1244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:16.804939 1253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:16.804955 1262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:16.804958 1256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:16.805174 1244 server_base.cc:1034] running on GCE node
I20260808 05:02:16.805460 1244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:16.805706 1244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:16.806839 1244 hybrid_clock.cc:648] HybridClock initialized: now 1786165336806821 us; error 33 us; skew 500 ppm
Aug 08 05:02:16 dist-test-slave-gjw7 krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165336, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:16.809211 1244 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:16.809634 1244 webserver.cc:460] Webserver started at http://127.3.174.190:46215/ using document root <none> and password file <none>
I20260808 05:02:16.809837 1244 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:16.809885 1244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:16.810009 1244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:16.811017 1244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "9f99d229c4964a16aa74976b5baa9b99"
format_stamp: "Formatted at 2026-08-08 05:02:16 on dist-test-slave-gjw7"
server_key: "2b4b72aafb6dcf1e9b79828adfb61d93"
server_key_iv: "c862a67b3eb28dbff8e9543e47958a9c"
server_key_version: "encryptionkey@0"
I20260808 05:02:16.811358 1244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "9f99d229c4964a16aa74976b5baa9b99"
format_stamp: "Formatted at 2026-08-08 05:02:16 on dist-test-slave-gjw7"
server_key: "2b4b72aafb6dcf1e9b79828adfb61d93"
server_key_iv: "c862a67b3eb28dbff8e9543e47958a9c"
server_key_version: "encryptionkey@0"
I20260808 05:02:16.812578 1244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 05:02:16.813210 1275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:16.813385 1244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:16.813472 1244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "9f99d229c4964a16aa74976b5baa9b99"
format_stamp: "Formatted at 2026-08-08 05:02:16 on dist-test-slave-gjw7"
server_key: "2b4b72aafb6dcf1e9b79828adfb61d93"
server_key_iv: "c862a67b3eb28dbff8e9543e47958a9c"
server_key_version: "encryptionkey@0"
I20260808 05:02:16.813534 1244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:16.823992 1244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:16.826108 1244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:16.826270 1244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:16.830175 1244 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:36683
I20260808 05:02:16.830217 1369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:36683 every 8 connection(s)
I20260808 05:02:16.830544 1244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:16.831403 1370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:16.833861 1370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99: Bootstrap starting.
I20260808 05:02:16.834415 1370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:16.834649 1370 log.cc:824] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:16.835285 1370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99: No bootstrap required, opened a new log
I20260808 05:02:16.836884 1370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f99d229c4964a16aa74976b5baa9b99" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36683 } }
I20260808 05:02:16.837023 1370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:16.837044 1370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f99d229c4964a16aa74976b5baa9b99, State: Initialized, Role: FOLLOWER
I20260808 05:02:16.837160 1370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f99d229c4964a16aa74976b5baa9b99" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36683 } }
I20260808 05:02:16.837239 1370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:16.837270 1370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:16.837332 1370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:16.837826 1370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f99d229c4964a16aa74976b5baa9b99" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36683 } }
I20260808 05:02:16.837913 1370 leader_election.cc:302] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f99d229c4964a16aa74976b5baa9b99; no voters:
I20260808 05:02:16.838032 1370 leader_election.cc:288] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:16.838074 1373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:16.838256 1373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [term 1 LEADER]: Becoming Leader. State: Replica: 9f99d229c4964a16aa74976b5baa9b99, State: Running, Role: LEADER
I20260808 05:02:16.838361 1373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f99d229c4964a16aa74976b5baa9b99" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36683 } }
I20260808 05:02:16.838389 1370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:16.838742 1375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f99d229c4964a16aa74976b5baa9b99. Latest consensus state: current_term: 1 leader_uuid: "9f99d229c4964a16aa74976b5baa9b99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f99d229c4964a16aa74976b5baa9b99" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36683 } } }
I20260808 05:02:16.838824 1375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:16.838850 1374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f99d229c4964a16aa74976b5baa9b99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f99d229c4964a16aa74976b5baa9b99" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36683 } } }
I20260808 05:02:16.838903 1374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:16.838987 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 1244
I20260808 05:02:16.839102 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:16.839248 1381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:16.839329 3770 external_mini_cluster.cc:1428] Setting key 01615880d147e534b153a8a0f59c37b9
I20260808 05:02:16.839803 1381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:16.841667 1381 catalog_manager.cc:1347] Generated new cluster ID: d458689ff47e46fda33dc2a04aacad67
I20260808 05:02:16.841720 1381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 05:02:16 dist-test-slave-gjw7 krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:16.848981 1381 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:16.849701 1381 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:16.849931 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:16.840142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.3.174.190:36683)
0808 05:02:16.840302 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:16.840313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:16.840346 (+ 33us) server_negotiation.cc:407] Connection header received
0808 05:02:16.840546 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:16.840553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:16.840623 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:16.840764 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:16.841569 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:16.842534 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:16.843267 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:16.843671 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:16.843766 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:16.846161 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:16.846165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:16.846165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:16.846178 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:16.848176 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:16.848556 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:16.848559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:16.848560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:16.848683 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:16.848969 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:16.848969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:16.848970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:16.849240 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:16.849434 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:16.849505 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 05:02:16.849693 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260808 05:02:16.858100 1381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99: Generated new TSK 0
I20260808 05:02:16.858279 1381 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 05:02:16.897768 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36683
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:17.006625 1418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:17.006840 1418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:17.006870 1418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:17.007993 1418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:17.008025 1418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:17.008050 1418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:02:17.009542 1418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:17.009727 1418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:17.009922 1418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:17.014676 1425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.014676 1434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.014690 1428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:17.014883 1418 server_base.cc:1034] running on GCE node
I20260808 05:02:17.015136 1418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:17.015374 1418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:17.016516 1418 hybrid_clock.cc:648] HybridClock initialized: now 1786165337016491 us; error 35 us; skew 500 ppm
Aug 08 05:02:17 dist-test-slave-gjw7 krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165337, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:17.018983 1418 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:02:17.019390 1418 webserver.cc:460] Webserver started at http://127.3.174.129:44065/ using document root <none> and password file <none>
I20260808 05:02:17.019586 1418 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:17.019629 1418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:17.019747 1418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:17.020816 1418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "f7db676f1a1926261b0b8a763dca0db7"
server_key_iv: "1265090efafb896becab73c346f84a42"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.021136 1418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "f7db676f1a1926261b0b8a763dca0db7"
server_key_iv: "1265090efafb896becab73c346f84a42"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.022535 1418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:02:17.023190 1449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.023356 1418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:17.023442 1418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "f7db676f1a1926261b0b8a763dca0db7"
server_key_iv: "1265090efafb896becab73c346f84a42"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.023510 1418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:17.034785 1418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:17.036542 1418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:17.036671 1418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:17.036872 1418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:17.037160 1418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:17.037194 1418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.037221 1418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:17.037240 1418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.042974 1418 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:33847
I20260808 05:02:17.043045 1643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:33847 every 8 connection(s)
I20260808 05:02:17.043319 1418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:02:17.044309 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 1418
I20260808 05:02:17.044418 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:02:17.044613 3770 external_mini_cluster.cc:1428] Setting key ddf14d4530330c0c3121a05c17e0279d
Aug 08 05:02:17 dist-test-slave-gjw7 krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165337, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:17.053357 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.044018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:57735 (local address 127.3.174.190:36683)
0808 05:02:17.044075 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.044077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.044253 (+ 176us) server_negotiation.cc:407] Connection header received
0808 05:02:17.044818 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.044819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.044832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.044888 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:17.045974 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.046676 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.047769 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.048068 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.048152 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.050334 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:17.050336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:17.050336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:17.050346 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:17.051937 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:17.052340 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:17.052342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:17.052342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:17.052407 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:17.052643 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:17.052644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:17.052644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:17.052772 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:17.052837 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.053117 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.053226 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 05:02:17.053499 1648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.043964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:36683 (local address 127.3.174.129:57735)
0808 05:02:17.044119 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.044159 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.044692 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.044974 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.044979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.045683 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:17.045872 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.045877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.046791 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.046792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.047674 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.047677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.048169 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.048170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.048285 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.048791 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:17.048800 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:17.050217 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:17.052022 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:17.052030 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:17.052032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:17.052265 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:17.052467 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:17.052468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:17.052469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:17.052581 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:17.052834 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:17.052842 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:17.052979 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.053086 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.053258 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
I20260808 05:02:17.053942 1645 heartbeater.cc:347] Connected to a master server at 127.3.174.190:36683
I20260808 05:02:17.054055 1645 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:17.054242 1645 heartbeater.cc:511] Master 127.3.174.190:36683 requested a full tablet report, sending...
I20260808 05:02:17.054759 1306 ts_manager.cc:194] Registered new tserver with Master: a8ad481e9b5b4c84bfbed76d0e3a866d (127.3.174.129:33847)
I20260808 05:02:17.055378 1306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:57735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 05:02:17.069839 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36683
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:17.153168 1655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:17.153394 1655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:17.153419 1655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:17.154597 1655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:17.154635 1655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:17.154659 1655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:02:17.156102 1655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:17.156301 1655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:17.156474 1655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:17.161347 1661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.161348 1662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.161433 1667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:17.161736 1655 server_base.cc:1034] running on GCE node
I20260808 05:02:17.161922 1655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:17.162110 1655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:17.163235 1655 hybrid_clock.cc:648] HybridClock initialized: now 1786165337163216 us; error 36 us; skew 500 ppm
Aug 08 05:02:17 dist-test-slave-gjw7 krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165337, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:17.165592 1655 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:02:17.166028 1655 webserver.cc:460] Webserver started at http://127.3.174.130:37479/ using document root <none> and password file <none>
I20260808 05:02:17.166224 1655 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:17.166265 1655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:17.166378 1655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:17.167383 1655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "ff1417364a024d5f8386c570fdc66833"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "8ba6e85ca690bb8c9a9f90fe7d5e886e"
server_key_iv: "0ae2b2ef794c7156135b39c64424f75d"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.167706 1655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "ff1417364a024d5f8386c570fdc66833"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "8ba6e85ca690bb8c9a9f90fe7d5e886e"
server_key_iv: "0ae2b2ef794c7156135b39c64424f75d"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.168959 1655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 05:02:17.169677 1681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.169827 1655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.169909 1655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "ff1417364a024d5f8386c570fdc66833"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "8ba6e85ca690bb8c9a9f90fe7d5e886e"
server_key_iv: "0ae2b2ef794c7156135b39c64424f75d"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.169973 1655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:17.186906 1655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:17.188582 1655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:17.188705 1655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:17.188927 1655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:17.189222 1655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:17.189256 1655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.189278 1655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:17.189330 1655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.195523 1655 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:35529
I20260808 05:02:17.195617 1886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:35529 every 8 connection(s)
I20260808 05:02:17.195936 1655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 05:02:17 dist-test-slave-gjw7 krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165337, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:17.204528 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.196676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:44085 (local address 127.3.174.190:36683)
0808 05:02:17.196736 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.196738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.196939 (+ 201us) server_negotiation.cc:407] Connection header received
0808 05:02:17.197921 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.197923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.197932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.197982 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:17.198544 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.199041 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.199947 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.200127 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.200172 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.202175 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:17.202176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:17.202176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:17.202184 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:17.203290 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:17.203653 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:17.203654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:17.203654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:17.203709 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:17.203925 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:17.203926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:17.203926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:17.204035 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:17.204092 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.204345 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.204438 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 05:02:17.204689 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.196650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:36683 (local address 127.3.174.130:44085)
0808 05:02:17.196821 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.196863 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.197839 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.198045 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.198048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.198289 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:17.198482 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.198486 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.199106 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.199107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.199887 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.199889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.200179 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.200179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.200275 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.200679 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:17.200687 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:17.202082 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:17.203366 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:17.203374 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:17.203376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:17.203585 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:17.203763 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:17.203764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:17.203764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:17.203868 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:17.204090 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:17.204097 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:17.204225 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.204322 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.204479 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260808 05:02:17.204890 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 1655
I20260808 05:02:17.204977 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:02:17.205168 3770 external_mini_cluster.cc:1428] Setting key a18cc2768cba91a6b0b5bad45774a244
I20260808 05:02:17.205339 1887 heartbeater.cc:347] Connected to a master server at 127.3.174.190:36683
I20260808 05:02:17.205451 1887 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:17.205648 1887 heartbeater.cc:511] Master 127.3.174.190:36683 requested a full tablet report, sending...
I20260808 05:02:17.206049 1306 ts_manager.cc:194] Registered new tserver with Master: ff1417364a024d5f8386c570fdc66833 (127.3.174.130:35529)
I20260808 05:02:17.206385 1306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:44085
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 05:02:17.229543 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36683
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:17.312789 1896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:17.313015 1896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:17.313042 1896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:17.314193 1896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:17.314236 1896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:17.314261 1896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:02:17.315738 1896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:17.315966 1896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:17.316150 1896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:17.321071 1908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.321112 1910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:17.321131 1896 server_base.cc:1034] running on GCE node
W20260808 05:02:17.321105 1914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:17.321496 1896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:17.321725 1896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:17.322866 1896 hybrid_clock.cc:648] HybridClock initialized: now 1786165337322846 us; error 33 us; skew 500 ppm
Aug 08 05:02:17 dist-test-slave-gjw7 krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165337, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:17.325248 1896 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:02:17.325711 1896 webserver.cc:460] Webserver started at http://127.3.174.131:34699/ using document root <none> and password file <none>
I20260808 05:02:17.325932 1896 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:17.325981 1896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:17.326102 1896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:17.327189 1896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "e147b2e570194cb5bae206bbcd2ea5ce"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "033bd7e052e0f868b5a804856ddbf41b"
server_key_iv: "cebd56dbe3073edf0d7c9366d946d249"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.327534 1896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "e147b2e570194cb5bae206bbcd2ea5ce"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "033bd7e052e0f868b5a804856ddbf41b"
server_key_iv: "cebd56dbe3073edf0d7c9366d946d249"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.328758 1896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 05:02:17.329437 1925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.329591 1896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.329674 1896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "e147b2e570194cb5bae206bbcd2ea5ce"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "033bd7e052e0f868b5a804856ddbf41b"
server_key_iv: "cebd56dbe3073edf0d7c9366d946d249"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.329742 1896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:17.350159 1896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:17.351941 1896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:17.352074 1896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:17.352283 1896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:17.352588 1896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:17.352622 1896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.352651 1896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:17.352675 1896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.358352 1896 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:33099
I20260808 05:02:17.358409 2116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:33099 every 8 connection(s)
I20260808 05:02:17.358721 1896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:02:17.364245 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 1896
Aug 08 05:02:17 dist-test-slave-gjw7 krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165337, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:17.364334 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 05:02:17.364498 3770 external_mini_cluster.cc:1428] Setting key 2911fdca78cad2429f822eaf47f1de31
I20260808 05:02:17.367323 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.359390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:56431 (local address 127.3.174.190:36683)
0808 05:02:17.359447 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.359449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.359640 (+ 191us) server_negotiation.cc:407] Connection header received
0808 05:02:17.360125 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.360126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.360137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.360185 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:17.361134 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.361667 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.362588 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.362771 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.362816 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.364839 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:17.364840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:17.364840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:17.364847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:17.366268 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:17.366631 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:17.366631 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:17.366632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:17.366676 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:17.366859 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:17.366859 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:17.366860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:17.366956 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:17.367000 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.367202 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.367256 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 05:02:17.367561 2120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.359377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:36683 (local address 127.3.174.131:56431)
0808 05:02:17.359535 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.359571 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.360049 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.360241 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.360244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.360865 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:17.361057 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.361062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.361730 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.361731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.362525 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.362527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.362829 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.362830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.362921 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.363346 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:17.363354 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:17.364756 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:17.366354 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:17.366361 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:17.366363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:17.366574 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:17.366719 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:17.366720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:17.366720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:17.366818 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:17.366997 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:17.367000 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:17.367109 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.367191 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.367334 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260808 05:02:17.368000 2117 heartbeater.cc:347] Connected to a master server at 127.3.174.190:36683
I20260808 05:02:17.368088 2117 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:17.368263 2117 heartbeater.cc:511] Master 127.3.174.190:36683 requested a full tablet report, sending...
I20260808 05:02:17.368613 1306 ts_manager.cc:194] Registered new tserver with Master: e147b2e570194cb5bae206bbcd2ea5ce (127.3.174.131:33099)
I20260808 05:02:17.368944 1306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:56431
I20260808 05:02:17.376142 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:02:17.382862 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.377414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.3.174.190:36683)
0808 05:02:17.377465 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.377467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.377576 (+ 109us) server_negotiation.cc:407] Connection header received
0808 05:02:17.377645 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.377646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.377656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.377702 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:17.378219 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.378691 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.379381 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.379579 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.379632 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.380681 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:17.380683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:17.380683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:17.380690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:17.381843 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:17.382140 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:17.382141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:17.382141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:17.382191 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:17.382354 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:17.382355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:17.382355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:17.382496 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:17.382586 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.382688 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.382764 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 05:02:17.384266 1306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:02:17.385890 1306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:02:17.392331 2139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.388096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:35529 (local address 127.0.0.1:44572)
0808 05:02:17.388256 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.388297 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.388359 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.388815 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.388816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.388818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:17.388874 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.388877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.389889 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.389890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.391134 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.391138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.392029 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.392030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.392108 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.392125 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.392189 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.392256 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":58,"threads_started":1}
I20260808 05:02:17.392508 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.388304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33847 (local address 127.0.0.1:56738)
0808 05:02:17.388474 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.388482 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.388517 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.388885 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.388886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.388888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:17.388919 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.388920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.390259 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.390260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.391319 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.391322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.392144 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.392145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.392209 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.392213 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.392266 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.392319 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260808 05:02:17.392590 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.388229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44572 (local address 127.3.174.130:35529)
0808 05:02:17.388640 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.388645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.388655 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:17.388693 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.388695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.388720 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.388791 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:17.388947 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.389794 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.391226 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.391944 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.392019 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.392080 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.392460 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.392510 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":79,"threads_started":1}
I20260808 05:02:17.392402 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.388530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.3.174.129:33847)
0808 05:02:17.388719 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.388723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.388733 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:17.388771 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.388773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.388787 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.388991 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:17.389077 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.390162 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.391389 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.392081 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.392140 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.392192 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.392281 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.392321 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260808 05:02:17.393103 2140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.388118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:33099 (local address 127.0.0.1:35254)
0808 05:02:17.388466 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.388475 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.388533 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.388772 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.388775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.388777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:17.388866 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.388868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.391387 (+ 2519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.391387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.392051 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.392053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.392886 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.392888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.392948 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.392952 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.393003 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.393041 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":84,"threads_started":1}
I20260808 05:02:17.393217 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.388276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35254 (local address 127.3.174.131:33099)
0808 05:02:17.388448 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.388453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.388521 (+ 68us) server_negotiation.cc:407] Connection header received
0808 05:02:17.388596 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.388598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.388614 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.388685 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:17.389000 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.390162 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.392143 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.392808 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.392871 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.392925 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.393024 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.393076 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260808 05:02:17.393419 1766 tablet_service.cc:1467] Processing CreateTablet for tablet 3b378683d7d048cfa75d88ec21fcd80e (DEFAULT_TABLE table=test-table [id=cbe1cff0afd2449c90072127bd3232c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:17.393666 1522 tablet_service.cc:1467] Processing CreateTablet for tablet 3b378683d7d048cfa75d88ec21fcd80e (DEFAULT_TABLE table=test-table [id=cbe1cff0afd2449c90072127bd3232c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:17.393819 1766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b378683d7d048cfa75d88ec21fcd80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:17.394021 1522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b378683d7d048cfa75d88ec21fcd80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:17.394018 2009 tablet_service.cc:1467] Processing CreateTablet for tablet 3b378683d7d048cfa75d88ec21fcd80e (DEFAULT_TABLE table=test-table [id=cbe1cff0afd2449c90072127bd3232c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:17.394320 2009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b378683d7d048cfa75d88ec21fcd80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:17.396682 2146 tablet_bootstrap.cc:492] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d: Bootstrap starting.
I20260808 05:02:17.397181 2150 tablet_bootstrap.cc:492] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: Bootstrap starting.
I20260808 05:02:17.397374 2146 tablet_bootstrap.cc:654] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:17.397708 2151 tablet_bootstrap.cc:492] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: Bootstrap starting.
I20260808 05:02:17.397945 2146 log.cc:824] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:17.398017 2150 tablet_bootstrap.cc:654] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:17.398337 2151 tablet_bootstrap.cc:654] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:17.398555 2150 log.cc:824] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:17.398744 2146 tablet_bootstrap.cc:492] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d: No bootstrap required, opened a new log
I20260808 05:02:17.398823 2146 ts_tablet_manager.cc:1397] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 05:02:17.398854 2151 log.cc:824] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:17.399245 2150 tablet_bootstrap.cc:492] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: No bootstrap required, opened a new log
I20260808 05:02:17.399318 2150 ts_tablet_manager.cc:1397] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:02:17.399768 2151 tablet_bootstrap.cc:492] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: No bootstrap required, opened a new log
I20260808 05:02:17.399854 2151 ts_tablet_manager.cc:1397] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:02:17.400774 2150 raft_consensus.cc:348] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } }
I20260808 05:02:17.400916 2150 raft_consensus.cc:374] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:17.400949 2150 raft_consensus.cc:729] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1417364a024d5f8386c570fdc66833, State: Initialized, Role: FOLLOWER
I20260808 05:02:17.400723 2146 raft_consensus.cc:348] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } }
I20260808 05:02:17.401090 2146 raft_consensus.cc:374] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:17.401100 2150 consensus_queue.cc:260] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } }
I20260808 05:02:17.401144 2146 raft_consensus.cc:729] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ad481e9b5b4c84bfbed76d0e3a866d, State: Initialized, Role: FOLLOWER
I20260808 05:02:17.401261 2146 consensus_queue.cc:260] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } }
I20260808 05:02:17.401424 1887 heartbeater.cc:503] Master 127.3.174.190:36683 was elected leader, sending a full tablet report...
I20260808 05:02:17.401629 1645 heartbeater.cc:503] Master 127.3.174.190:36683 was elected leader, sending a full tablet report...
I20260808 05:02:17.401722 2150 ts_tablet_manager.cc:1428] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:02:17.401960 2146 ts_tablet_manager.cc:1428] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 05:02:17.402096 2151 raft_consensus.cc:348] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } }
I20260808 05:02:17.402254 2151 raft_consensus.cc:374] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:17.402298 2151 raft_consensus.cc:729] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e147b2e570194cb5bae206bbcd2ea5ce, State: Initialized, Role: FOLLOWER
I20260808 05:02:17.402417 2151 consensus_queue.cc:260] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } }
I20260808 05:02:17.402690 2151 ts_tablet_manager.cc:1428] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 05:02:17.402743 2117 heartbeater.cc:503] Master 127.3.174.190:36683 was elected leader, sending a full tablet report...
I20260808 05:02:17.417659 2163 raft_consensus.cc:482] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:17.417778 2163 raft_consensus.cc:504] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } }
I20260808 05:02:17.418118 2163 leader_election.cc:288] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff1417364a024d5f8386c570fdc66833 (127.3.174.130:35529), a8ad481e9b5b4c84bfbed76d0e3a866d (127.3.174.129:33847)
I20260808 05:02:17.420682 2120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.418267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:35529 (local address 127.3.174.131:38059)
0808 05:02:17.418322 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.418335 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.418384 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.418547 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.418548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.418549 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:17.418610 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.418612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.419188 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.419189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.419924 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.419926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.420441 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.420442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.420522 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.420525 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.420569 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.420611 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 05:02:17.420682 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.418339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:38059 (local address 127.3.174.130:35529)
0808 05:02:17.418388 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.418391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.418418 (+ 27us) server_negotiation.cc:407] Connection header received
0808 05:02:17.418445 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.418447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.418458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.418502 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:17.418667 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.419122 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.420002 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.420373 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.420435 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.420444 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.420580 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.420613 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 05:02:17.420907 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.418462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:35133 (local address 127.3.174.129:33847)
0808 05:02:17.418511 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.418513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.418597 (+ 84us) server_negotiation.cc:407] Connection header received
0808 05:02:17.418657 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.418658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.418669 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.418717 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:17.418851 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.419315 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.420205 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.420574 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.420636 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.420644 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.420797 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.420843 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 05:02:17.421047 1803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b378683d7d048cfa75d88ec21fcd80e" candidate_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1417364a024d5f8386c570fdc66833" is_pre_election: true
I20260808 05:02:17.421125 2166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.418374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33847 (local address 127.3.174.131:35133)
0808 05:02:17.418543 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.418551 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.418600 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.418725 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.418726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.418727 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:17.418794 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.418796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.419391 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.419392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.420138 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.420140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.420648 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.420649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.420711 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.420714 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.421016 (+ 302us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.421055 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":111,"threads_started":1}
I20260808 05:02:17.421191 1803 raft_consensus.cc:2454] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e147b2e570194cb5bae206bbcd2ea5ce in term 0.
I20260808 05:02:17.421443 1928 leader_election.cc:302] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e147b2e570194cb5bae206bbcd2ea5ce, ff1417364a024d5f8386c570fdc66833; no voters:
I20260808 05:02:17.421571 1564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b378683d7d048cfa75d88ec21fcd80e" candidate_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" is_pre_election: true
I20260808 05:02:17.421586 2163 raft_consensus.cc:2781] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:02:17.421648 2163 raft_consensus.cc:482] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:02:17.421670 2163 raft_consensus.cc:3037] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:17.421694 1564 raft_consensus.cc:2454] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e147b2e570194cb5bae206bbcd2ea5ce in term 0.
I20260808 05:02:17.422411 2163 raft_consensus.cc:504] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } }
I20260808 05:02:17.422526 2163 leader_election.cc:288] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [CANDIDATE]: Term 1 election: Requested vote from peers ff1417364a024d5f8386c570fdc66833 (127.3.174.130:35529), a8ad481e9b5b4c84bfbed76d0e3a866d (127.3.174.129:33847)
I20260808 05:02:17.422698 1564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b378683d7d048cfa75d88ec21fcd80e" candidate_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d"
I20260808 05:02:17.422689 1803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b378683d7d048cfa75d88ec21fcd80e" candidate_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1417364a024d5f8386c570fdc66833"
I20260808 05:02:17.422770 1564 raft_consensus.cc:3037] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:17.422770 1803 raft_consensus.cc:3037] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:17.423642 1803 raft_consensus.cc:2454] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e147b2e570194cb5bae206bbcd2ea5ce in term 1.
I20260808 05:02:17.423642 1564 raft_consensus.cc:2454] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e147b2e570194cb5bae206bbcd2ea5ce in term 1.
I20260808 05:02:17.423854 1928 leader_election.cc:302] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e147b2e570194cb5bae206bbcd2ea5ce, ff1417364a024d5f8386c570fdc66833; no voters:
I20260808 05:02:17.423966 2163 raft_consensus.cc:2781] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:17.424139 2163 raft_consensus.cc:686] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 1 LEADER]: Becoming Leader. State: Replica: e147b2e570194cb5bae206bbcd2ea5ce, State: Running, Role: LEADER
I20260808 05:02:17.424299 2163 consensus_queue.cc:237] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } }
I20260808 05:02:17.425230 1306 catalog_manager.cc:5495] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce reported cstate change: term changed from 0 to 1, leader changed from <none> to e147b2e570194cb5bae206bbcd2ea5ce (127.3.174.131). New cstate: current_term: 1 leader_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e147b2e570194cb5bae206bbcd2ea5ce" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 33099 } health_report { overall_health: HEALTHY } } }
I20260808 05:02:17.437986 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.435885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35258 (local address 127.3.174.131:33099)
0808 05:02:17.435941 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.435943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.435950 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:02:17.436012 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.436012 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.436027 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.436081 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:17.436185 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.436623 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.437408 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.437586 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.437634 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.437737 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:17.437835 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:17.437886 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.437901 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.437917 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 05:02:17.439779 1564 raft_consensus.cc:1261] T 3b378683d7d048cfa75d88ec21fcd80e P a8ad481e9b5b4c84bfbed76d0e3a866d [term 1 FOLLOWER]: Refusing update from remote peer e147b2e570194cb5bae206bbcd2ea5ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:17.439779 1803 raft_consensus.cc:1261] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [term 1 FOLLOWER]: Refusing update from remote peer e147b2e570194cb5bae206bbcd2ea5ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:17.440107 2168 consensus_queue.cc:1035] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1417364a024d5f8386c570fdc66833" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 35529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:17.440232 2163 consensus_queue.cc:1035] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ad481e9b5b4c84bfbed76d0e3a866d" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:17.443027 2176 mvcc.cc:205] Tried to move back new op lower bound from 7316133222150770688 to 7316133222090739712. Current Snapshot: MvccSnapshot[applied={T|T < 7316133222150770688}]
I20260808 05:02:17.443646 2175 mvcc.cc:205] Tried to move back new op lower bound from 7316133222150770688 to 7316133222090739712. Current Snapshot: MvccSnapshot[applied={T|T < 7316133222150770688 or (T in {7316133222150770688})}]
I20260808 05:02:17.444814 1306 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:17.444896 1306 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:17.445614 1306 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f99d229c4964a16aa74976b5baa9b99: Sending DeleteTablet for 3 replicas of tablet 3b378683d7d048cfa75d88ec21fcd80e
I20260808 05:02:17.445922 2009 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b378683d7d048cfa75d88ec21fcd80e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:17 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:35254
I20260808 05:02:17.445935 1766 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b378683d7d048cfa75d88ec21fcd80e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:17 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:44572
I20260808 05:02:17.445981 1522 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b378683d7d048cfa75d88ec21fcd80e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:17 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:56738
I20260808 05:02:17.446161 2189 tablet_replica.cc:330] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: stopping tablet replica
I20260808 05:02:17.446278 2187 tablet_replica.cc:330] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: stopping tablet replica
I20260808 05:02:17.446393 2187 raft_consensus.cc:2229] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:02:17.446431 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 1418
I20260808 05:02:17.468585 2189 raft_consensus.cc:2229] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:02:17.468849 2189 raft_consensus.cc:2258] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:17.471453 2189 ts_tablet_manager.cc:1905] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:17.475493 2189 ts_tablet_manager.cc:1918] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:17.475582 2189 log.cc:1196] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/3b378683d7d048cfa75d88ec21fcd80e
I20260808 05:02:17.475836 2189 ts_tablet_manager.cc:1939] T 3b378683d7d048cfa75d88ec21fcd80e P e147b2e570194cb5bae206bbcd2ea5ce: Deleting consensus metadata
I20260808 05:02:17.476331 1284 catalog_manager.cc:4841] TS e147b2e570194cb5bae206bbcd2ea5ce (127.3.174.131:33099): tablet 3b378683d7d048cfa75d88ec21fcd80e (table test-table [id=cbe1cff0afd2449c90072127bd3232c6]) successfully deleted
I20260808 05:02:17.446494 2187 raft_consensus.cc:2258] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:17.478081 2187 ts_tablet_manager.cc:1905] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:17.479408 2187 ts_tablet_manager.cc:1918] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:17.479478 2187 log.cc:1196] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/3b378683d7d048cfa75d88ec21fcd80e
I20260808 05:02:17.479703 2187 ts_tablet_manager.cc:1939] T 3b378683d7d048cfa75d88ec21fcd80e P ff1417364a024d5f8386c570fdc66833: Deleting consensus metadata
I20260808 05:02:17.480137 1282 catalog_manager.cc:4841] TS ff1417364a024d5f8386c570fdc66833 (127.3.174.130:35529): tablet 3b378683d7d048cfa75d88ec21fcd80e (table test-table [id=cbe1cff0afd2449c90072127bd3232c6]) successfully deleted
W20260808 05:02:17.480861 1282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:33847 (error 108)
I20260808 05:02:17.481099 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 1655
I20260808 05:02:17.481437 2140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.481282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33847 (local address 127.0.0.1:56740)
0808 05:02:17.481364 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.481402 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 05:02:17.481554 1282 catalog_manager.cc:4584] TS a8ad481e9b5b4c84bfbed76d0e3a866d (127.3.174.129:33847): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b378683d7d048cfa75d88ec21fcd80e: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33847: connect: Connection refused (error 111)
I20260808 05:02:17.492856 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 1896
I20260808 05:02:17.502413 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 1244
2026-08-08T05:02:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2105 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T05:02:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:17Z Disabled control of system clock
I20260808 05:02:17.538074 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:42031
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:42031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:17.620702 2201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:17.620929 2201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:17.620947 2201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:17.622084 2201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:17.622120 2201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:17.622131 2201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:17.622143 2201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:17.622153 2201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:17.623456 2201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:42031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:42031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:17.623654 2201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:17.623901 2201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:17.628566 2212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.628585 2210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.628579 2209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:17.628639 2201 server_base.cc:1034] running on GCE node
I20260808 05:02:17.629101 2201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:17.629415 2201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:17.630573 2201 hybrid_clock.cc:648] HybridClock initialized: now 1786165337630554 us; error 30 us; skew 500 ppm
I20260808 05:02:17.631834 2201 webserver.cc:460] Webserver started at http://127.3.174.190:33991/ using document root <none> and password file <none>
I20260808 05:02:17.632053 2201 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:17.632102 2201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:17.632232 2201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:17.633324 2201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "56a95209a574496da22e51dc75527551"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "60b66281eb88e9ae28638f7d2860c1c5"
server_key_iv: "2dded2a5614e41d731d4a345b2a2274f"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.633684 2201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "56a95209a574496da22e51dc75527551"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "60b66281eb88e9ae28638f7d2860c1c5"
server_key_iv: "2dded2a5614e41d731d4a345b2a2274f"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.635079 2201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:17.635855 2223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.636035 2201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:17.636132 2201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "56a95209a574496da22e51dc75527551"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "60b66281eb88e9ae28638f7d2860c1c5"
server_key_iv: "2dded2a5614e41d731d4a345b2a2274f"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.636209 2201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:17.650132 2201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:17.650440 2201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:17.650579 2201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:17.654882 2201 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:42031
I20260808 05:02:17.654915 2315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:42031 every 8 connection(s)
I20260808 05:02:17.655268 2201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:17.656307 2316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:17.658743 2316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551: Bootstrap starting.
I20260808 05:02:17.659322 2316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:17.659623 2316 log.cc:824] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:17.660308 2316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551: No bootstrap required, opened a new log
I20260808 05:02:17.661954 2316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a95209a574496da22e51dc75527551" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42031 } }
I20260808 05:02:17.662094 2316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:17.662117 2316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56a95209a574496da22e51dc75527551, State: Initialized, Role: FOLLOWER
I20260808 05:02:17.662302 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 2201
I20260808 05:02:17.662277 2316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a95209a574496da22e51dc75527551" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42031 } }
I20260808 05:02:17.662366 2316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:17.662400 2316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:17.662411 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:17.662439 2316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:17.662603 3770 external_mini_cluster.cc:1428] Setting key 4a9c48abc1a2c3840249a557024aebef
I20260808 05:02:17.663110 2316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a95209a574496da22e51dc75527551" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42031 } }
I20260808 05:02:17.663225 2316 leader_election.cc:302] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56a95209a574496da22e51dc75527551; no voters:
I20260808 05:02:17.663394 2316 leader_election.cc:288] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:17.663421 2322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:17.663583 2322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [term 1 LEADER]: Becoming Leader. State: Replica: 56a95209a574496da22e51dc75527551, State: Running, Role: LEADER
I20260808 05:02:17.663652 2316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:17.663702 2322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a95209a574496da22e51dc75527551" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42031 } }
I20260808 05:02:17.664179 2325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56a95209a574496da22e51dc75527551. Latest consensus state: current_term: 1 leader_uuid: "56a95209a574496da22e51dc75527551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a95209a574496da22e51dc75527551" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42031 } } }
I20260808 05:02:17.664284 2325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:17.664434 2324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56a95209a574496da22e51dc75527551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a95209a574496da22e51dc75527551" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 42031 } } }
I20260808 05:02:17.664479 2324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:17.664784 2336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:17.665350 2336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:17.666957 2336 catalog_manager.cc:1347] Generated new cluster ID: 73893f42936642089d92323194fae045
I20260808 05:02:17.667053 2336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:17.669023 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.663105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45484 (local address 127.3.174.190:42031)
0808 05:02:17.663428 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.663438 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.663472 (+ 34us) server_negotiation.cc:407] Connection header received
0808 05:02:17.663600 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.663608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.663683 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.663838 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:17.664057 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.665903 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.666881 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.667497 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.667629 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.668317 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:17.668322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:17.668322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:17.668537 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:17.668580 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:17.668596 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:17.668666 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.668727 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.668811 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":173,"threads_started":1}
I20260808 05:02:17.677874 2336 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:17.678558 2336 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:17.685721 2336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551: Generated new TSK 0
I20260808 05:02:17.685909 2336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:17.691891 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42031
--builtin_ntp_servers=127.3.174.148:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:17.776225 2355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:17.776434 2355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:17.776461 2355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:17.777673 2355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:17.777719 2355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:17.777751 2355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:02:17.779084 2355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:17.779323 2355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:17.779611 2355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:17.784544 2370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.784569 2366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.784569 2367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:17.784718 2355 server_base.cc:1034] running on GCE node
I20260808 05:02:17.785027 2355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:17.785256 2355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:17.786429 2355 hybrid_clock.cc:648] HybridClock initialized: now 1786165337786415 us; error 28 us; skew 500 ppm
I20260808 05:02:17.787708 2355 webserver.cc:460] Webserver started at http://127.3.174.129:45327/ using document root <none> and password file <none>
I20260808 05:02:17.787915 2355 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:17.787963 2355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:17.788089 2355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:17.789140 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "82877d98250246b4aa693c6aa0fc1390"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "8455cf2b7e496670c023784028058fc7"
server_key_iv: "9b8e20677baa15a3c9e935c81b5a948f"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.789574 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "82877d98250246b4aa693c6aa0fc1390"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "8455cf2b7e496670c023784028058fc7"
server_key_iv: "9b8e20677baa15a3c9e935c81b5a948f"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.790865 2355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:17.791570 2378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.791733 2355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:17.791813 2355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "82877d98250246b4aa693c6aa0fc1390"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "8455cf2b7e496670c023784028058fc7"
server_key_iv: "9b8e20677baa15a3c9e935c81b5a948f"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.791882 2355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:17.801043 2355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:17.801353 2355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:17.801481 2355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:17.801678 2355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:17.801981 2355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:17.802014 2355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.802042 2355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:17.802066 2355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.807585 2355 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:41839
I20260808 05:02:17.807651 2572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:41839 every 8 connection(s)
I20260808 05:02:17.807942 2355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:02:17.812681 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.808609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:41603 (local address 127.3.174.190:42031)
0808 05:02:17.808663 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.808665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.808870 (+ 205us) server_negotiation.cc:407] Connection header received
0808 05:02:17.809401 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.809402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.809414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.809466 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:17.810094 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.810586 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.811501 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.811696 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.811748 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.812450 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:17.812451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:17.812452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:17.812459 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:17.812472 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:17.812476 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:17.812519 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.812596 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.812622 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 05:02:17.812885 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.808566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:42031 (local address 127.3.174.129:41603)
0808 05:02:17.808758 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.808800 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.809275 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.809530 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.809534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.809543 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:17.810030 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.810036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.810648 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.810648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.811425 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.811427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.811755 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.811756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.811856 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.812317 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 05:02:17.812318 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:17.812347 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 05:02:17.812370 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:17.812533 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:17.812539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:17.812541 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.812592 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.812675 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260808 05:02:17.813401 2575 heartbeater.cc:347] Connected to a master server at 127.3.174.190:42031
I20260808 05:02:17.813491 2575 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:17.813669 2575 heartbeater.cc:511] Master 127.3.174.190:42031 requested a full tablet report, sending...
I20260808 05:02:17.814168 2252 ts_manager.cc:194] Registered new tserver with Master: 82877d98250246b4aa693c6aa0fc1390 (127.3.174.129:41839)
I20260808 05:02:17.814868 2252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.129:41603
I20260808 05:02:17.817638 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 2355
I20260808 05:02:17.817756 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:02:17.817940 3770 external_mini_cluster.cc:1428] Setting key ae7fe50154634c5aea09526a022fa5ed
I20260808 05:02:17.818837 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42031
--builtin_ntp_servers=127.3.174.148:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:17.904016 2580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:17.904213 2580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:17.904233 2580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:17.905400 2580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:17.905438 2580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:17.905463 2580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:02:17.906809 2580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:17.907028 2580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:17.907297 2580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:17.912103 2591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:17.912295 2580 server_base.cc:1034] running on GCE node
W20260808 05:02:17.912111 2593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:17.912124 2590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:17.912566 2580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:17.912792 2580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:17.913930 2580 hybrid_clock.cc:648] HybridClock initialized: now 1786165337913913 us; error 28 us; skew 500 ppm
I20260808 05:02:17.915108 2580 webserver.cc:460] Webserver started at http://127.3.174.130:43789/ using document root <none> and password file <none>
I20260808 05:02:17.915328 2580 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:17.915377 2580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:17.915511 2580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:17.916617 2580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "601fed68fee24fc0a4da4806b52ddde4"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "0395510785698b54d2a2cfa13486be04"
server_key_iv: "a1780d2687a0add7560bf2a8994ee5a4"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.916970 2580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "601fed68fee24fc0a4da4806b52ddde4"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "0395510785698b54d2a2cfa13486be04"
server_key_iv: "a1780d2687a0add7560bf2a8994ee5a4"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.918216 2580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:17.918867 2601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.919023 2580 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:17.919107 2580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "601fed68fee24fc0a4da4806b52ddde4"
format_stamp: "Formatted at 2026-08-08 05:02:17 on dist-test-slave-gjw7"
server_key: "0395510785698b54d2a2cfa13486be04"
server_key_iv: "a1780d2687a0add7560bf2a8994ee5a4"
server_key_version: "encryptionkey@0"
I20260808 05:02:17.919166 2580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:17.941061 2580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:17.941399 2580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:17.941537 2580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:17.941767 2580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:17.942106 2580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:17.942142 2580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.942171 2580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:17.942189 2580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:17.947633 2580 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:36817
I20260808 05:02:17.947680 2824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:36817 every 8 connection(s)
I20260808 05:02:17.948021 2580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 05:02:17.953258 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.948769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:33093 (local address 127.3.174.190:42031)
0808 05:02:17.948830 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:17.948832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:17.949063 (+ 231us) server_negotiation.cc:407] Connection header received
0808 05:02:17.949676 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:17.949678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:17.949694 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:17.949744 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:17.950431 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.950933 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.951896 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.952096 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.952154 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.952972 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:17.952974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:17.952975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:17.952981 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:17.952996 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:17.953001 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:17.953051 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:17.953125 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 05:02:17.953162 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 05:02:17.953470 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:17.948746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:42031 (local address 127.3.174.130:33093)
0808 05:02:17.948935 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:17.948972 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:17.949550 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:17.949826 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:17.949830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:17.949840 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:17.950339 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.950344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.951022 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.951023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.951813 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:17.951816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:17.952169 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:17.952171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:17.952304 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:17.952800 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 05:02:17.952801 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:17.952833 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 05:02:17.952859 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:17.953053 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:17.953059 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:17.953062 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:02:17.953124 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 05:02:17.953233 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260808 05:02:17.953971 2825 heartbeater.cc:347] Connected to a master server at 127.3.174.190:42031
I20260808 05:02:17.954066 2825 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:17.954229 2825 heartbeater.cc:511] Master 127.3.174.190:42031 requested a full tablet report, sending...
I20260808 05:02:17.954595 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 2580
I20260808 05:02:17.954636 2252 ts_manager.cc:194] Registered new tserver with Master: 601fed68fee24fc0a4da4806b52ddde4 (127.3.174.130:36817)
I20260808 05:02:17.954703 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:02:17.954893 3770 external_mini_cluster.cc:1428] Setting key 29bf7b2daf43a17ef888e58b1eac942e
I20260808 05:02:17.955051 2252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.130:33093
I20260808 05:02:17.955834 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42031
--builtin_ntp_servers=127.3.174.148:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:18.039546 2831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:18.039777 2831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:18.039811 2831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:18.040997 2831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:18.041039 2831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:18.041072 2831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:02:18.042454 2831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:42031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:18.042706 2831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:18.042990 2831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:18.048007 2840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:18.047988 2839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:18.047988 2843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:18.048135 2831 server_base.cc:1034] running on GCE node
I20260808 05:02:18.048425 2831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:18.048645 2831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:18.049772 2831 hybrid_clock.cc:648] HybridClock initialized: now 1786165338049760 us; error 27 us; skew 500 ppm
I20260808 05:02:18.050930 2831 webserver.cc:460] Webserver started at http://127.3.174.131:36063/ using document root <none> and password file <none>
I20260808 05:02:18.051124 2831 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:18.051164 2831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:18.051259 2831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:18.052312 2831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "bea503ffbbf44851a351354706b58eaa"
format_stamp: "Formatted at 2026-08-08 05:02:18 on dist-test-slave-gjw7"
server_key: "7c1626bc8c0b7202029121e7c0018c36"
server_key_iv: "c5a248d08f2719046fdfd7697ff8fbf8"
server_key_version: "encryptionkey@0"
I20260808 05:02:18.052618 2831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "bea503ffbbf44851a351354706b58eaa"
format_stamp: "Formatted at 2026-08-08 05:02:18 on dist-test-slave-gjw7"
server_key: "7c1626bc8c0b7202029121e7c0018c36"
server_key_iv: "c5a248d08f2719046fdfd7697ff8fbf8"
server_key_version: "encryptionkey@0"
I20260808 05:02:18.053929 2831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:18.054675 2855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:18.054880 2831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:18.054970 2831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "bea503ffbbf44851a351354706b58eaa"
format_stamp: "Formatted at 2026-08-08 05:02:18 on dist-test-slave-gjw7"
server_key: "7c1626bc8c0b7202029121e7c0018c36"
server_key_iv: "c5a248d08f2719046fdfd7697ff8fbf8"
server_key_version: "encryptionkey@0"
I20260808 05:02:18.055030 2831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:18.067543 2831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:18.067816 2831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:18.067916 2831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:18.068099 2831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:18.068377 2831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:18.068406 2831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:18.068434 2831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:18.068450 2831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:18.073895 2831 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:34801
I20260808 05:02:18.073961 3047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:34801 every 8 connection(s)
I20260808 05:02:18.074272 2831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:02:18.079159 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.074959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:52641 (local address 127.3.174.190:42031)
0808 05:02:18.075010 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:18.075013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:18.075303 (+ 290us) server_negotiation.cc:407] Connection header received
0808 05:02:18.075801 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:18.075802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:18.075813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:18.075858 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:18.076577 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.077071 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.078026 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.078212 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.078260 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.078948 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:18.078949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:18.078949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:18.078953 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:18.078967 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:18.078971 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:18.079010 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:18.079071 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 05:02:18.079097 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 05:02:18.079330 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.074948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:42031 (local address 127.3.174.131:52641)
0808 05:02:18.075178 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:18.075214 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:18.075699 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:18.075933 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:18.075936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:18.075946 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:18.076502 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.076508 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.077138 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.077138 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.077961 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.077964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.078272 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.078272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.078377 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.078830 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 05:02:18.078830 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:18.078858 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 05:02:18.078881 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:18.079015 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:18.079018 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:18.079021 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:02:18.079067 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 05:02:18.079143 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":96,"threads_started":1}
I20260808 05:02:18.079775 3048 heartbeater.cc:347] Connected to a master server at 127.3.174.190:42031
I20260808 05:02:18.079881 3048 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:18.080058 3048 heartbeater.cc:511] Master 127.3.174.190:42031 requested a full tablet report, sending...
I20260808 05:02:18.080436 2252 ts_manager.cc:194] Registered new tserver with Master: bea503ffbbf44851a351354706b58eaa (127.3.174.131:34801)
I20260808 05:02:18.080776 2252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.131:52641
I20260808 05:02:18.081239 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 2831
I20260808 05:02:18.081396 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 05:02:18.081574 3770 external_mini_cluster.cc:1428] Setting key 563c0c96a621582828bb0bcdea2ba61c
I20260808 05:02:18.082520 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:02:18.086293 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.083894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45496 (local address 127.3.174.190:42031)
0808 05:02:18.083938 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:18.083940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:18.084029 (+ 89us) server_negotiation.cc:407] Connection header received
0808 05:02:18.084093 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:18.084093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:18.084104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:18.084140 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:18.084268 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.084704 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.085423 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.085595 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.085637 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.086128 (+ 491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:18.086129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:18.086129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:18.086133 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:18.086144 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:18.086147 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:18.086185 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:18.086221 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 05:02:18.086240 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 05:02:18.087647 2252 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:02:18.089166 2252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:02:18.095209 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.091456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:41839 (local address 127.0.0.1:44432)
0808 05:02:18.091588 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:18.091599 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:18.091636 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:18.091993 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:18.091994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:18.091996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:18.092027 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.092028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.092913 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.092914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.094070 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.094073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.094918 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.094920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.094992 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.095005 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 05:02:18.095074 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 05:02:18.095127 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260808 05:02:18.095212 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.091668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44432 (local address 127.3.174.129:41839)
0808 05:02:18.091826 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:18.091831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:18.091840 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:18.091876 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:18.091878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:18.091893 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:18.092089 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:18.092148 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.092835 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.094166 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.094846 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.094907 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.095002 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:18.095091 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 05:02:18.095130 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260808 05:02:18.095534 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.091232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:34801 (local address 127.0.0.1:51594)
0808 05:02:18.091577 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:18.091586 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:18.091624 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:18.092145 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:18.092146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:18.092148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:18.092189 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.092190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.093433 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.093434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.094290 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.094294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.095313 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.095314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.095374 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.095378 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:18.095422 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 05:02:18.095459 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260808 05:02:18.095780 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.091232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:36817 (local address 127.0.0.1:33462)
0808 05:02:18.091378 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:18.091422 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:18.091486 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:18.091860 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:18.091863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:18.091866 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:18.091934 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.091935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.093315 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.093316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.094351 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.094354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.095572 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.095574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.095631 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.095635 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:18.095680 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 05:02:18.095718 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260808 05:02:18.095801 3067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.091308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.3.174.131:34801)
0808 05:02:18.091870 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:18.091874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:18.091884 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:18.091918 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:18.091920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:18.091934 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:18.092251 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:18.092320 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.093363 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.094363 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.095118 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.095182 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.095264 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:18.095685 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 05:02:18.095725 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":87,"threads_started":1}
I20260808 05:02:18.096071 3069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.091496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.3.174.130:36817)
0808 05:02:18.091675 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:18.091681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:18.091691 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:18.091733 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:18.091735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:18.091751 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:18.091850 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:18.092027 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.093199 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.094413 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.095118 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.095182 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.095264 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:18.095975 (+ 711us) server_negotiation.cc:299] Negotiation successful
0808 05:02:18.096007 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260808 05:02:18.096107 2461 tablet_service.cc:1467] Processing CreateTablet for tablet 77e426392cd0476ea66e5e1c11f39735 (DEFAULT_TABLE table=test-table [id=e9b28f2f2f6b4d7d84a6226a3f84a867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:18.096536 2461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77e426392cd0476ea66e5e1c11f39735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:18.096720 2701 tablet_service.cc:1467] Processing CreateTablet for tablet 77e426392cd0476ea66e5e1c11f39735 (DEFAULT_TABLE table=test-table [id=e9b28f2f2f6b4d7d84a6226a3f84a867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:18.097070 2701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77e426392cd0476ea66e5e1c11f39735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:18.097842 2944 tablet_service.cc:1467] Processing CreateTablet for tablet 77e426392cd0476ea66e5e1c11f39735 (DEFAULT_TABLE table=test-table [id=e9b28f2f2f6b4d7d84a6226a3f84a867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:18.098130 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77e426392cd0476ea66e5e1c11f39735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:18.100033 3074 tablet_bootstrap.cc:492] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390: Bootstrap starting.
I20260808 05:02:18.100322 3076 tablet_bootstrap.cc:492] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: Bootstrap starting.
I20260808 05:02:18.100859 3078 tablet_bootstrap.cc:492] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: Bootstrap starting.
I20260808 05:02:18.101058 3074 tablet_bootstrap.cc:654] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:18.101138 3076 tablet_bootstrap.cc:654] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:18.101409 3074 log.cc:824] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:18.101511 3078 tablet_bootstrap.cc:654] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:18.101495 3076 log.cc:824] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:18.101809 3078 log.cc:824] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:18.102156 3074 tablet_bootstrap.cc:492] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390: No bootstrap required, opened a new log
I20260808 05:02:18.102193 3076 tablet_bootstrap.cc:492] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: No bootstrap required, opened a new log
I20260808 05:02:18.102221 3074 ts_tablet_manager.cc:1397] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 05:02:18.102254 3076 ts_tablet_manager.cc:1397] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 05:02:18.102479 3078 tablet_bootstrap.cc:492] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: No bootstrap required, opened a new log
I20260808 05:02:18.102531 3078 ts_tablet_manager.cc:1397] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:02:18.103963 3074 raft_consensus.cc:348] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } }
I20260808 05:02:18.104084 3074 raft_consensus.cc:374] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:18.104102 3074 raft_consensus.cc:729] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82877d98250246b4aa693c6aa0fc1390, State: Initialized, Role: FOLLOWER
I20260808 05:02:18.104197 3074 consensus_queue.cc:260] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } }
I20260808 05:02:18.104442 3074 ts_tablet_manager.cc:1428] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 05:02:18.104475 2575 heartbeater.cc:503] Master 127.3.174.190:42031 was elected leader, sending a full tablet report...
I20260808 05:02:18.104776 3076 raft_consensus.cc:348] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } }
I20260808 05:02:18.104938 3076 raft_consensus.cc:374] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:18.105008 3076 raft_consensus.cc:729] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 601fed68fee24fc0a4da4806b52ddde4, State: Initialized, Role: FOLLOWER
I20260808 05:02:18.105132 3076 consensus_queue.cc:260] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } }
I20260808 05:02:18.105321 3078 raft_consensus.cc:348] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } }
I20260808 05:02:18.105453 3076 ts_tablet_manager.cc:1428] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 05:02:18.105484 2825 heartbeater.cc:503] Master 127.3.174.190:42031 was elected leader, sending a full tablet report...
I20260808 05:02:18.105504 3078 raft_consensus.cc:374] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:18.105543 3078 raft_consensus.cc:729] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea503ffbbf44851a351354706b58eaa, State: Initialized, Role: FOLLOWER
I20260808 05:02:18.105639 3078 consensus_queue.cc:260] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } }
I20260808 05:02:18.105945 3048 heartbeater.cc:503] Master 127.3.174.190:42031 was elected leader, sending a full tablet report...
I20260808 05:02:18.105927 3078 ts_tablet_manager.cc:1428] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260808 05:02:18.198594 2827 tablet.cc:2369] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 05:02:18.308638 2576 tablet.cc:2369] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 05:02:18.324796 3050 tablet.cc:2369] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:02:18.358388 3086 raft_consensus.cc:482] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:18.358539 3086 raft_consensus.cc:504] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } }
I20260808 05:02:18.358973 3086 leader_election.cc:288] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 601fed68fee24fc0a4da4806b52ddde4 (127.3.174.130:36817), 82877d98250246b4aa693c6aa0fc1390 (127.3.174.129:41839)
I20260808 05:02:18.362852 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.359285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:37309 (local address 127.3.174.129:41839)
0808 05:02:18.359327 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:18.359329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:18.359341 (+ 12us) server_negotiation.cc:407] Connection header received
0808 05:02:18.359427 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:18.359428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:18.359444 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:18.359506 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:18.359687 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.360456 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.361958 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.362531 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.362593 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.362599 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:18.362728 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 05:02:18.362769 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 05:02:18.362852 3069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.359310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:39741 (local address 127.3.174.130:36817)
0808 05:02:18.359358 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:18.359360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:18.359556 (+ 196us) server_negotiation.cc:407] Connection header received
0808 05:02:18.359620 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:18.359621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:18.359627 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:18.359671 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:18.359783 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.360470 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.361950 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.362492 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.362572 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.362581 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:18.362745 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 05:02:18.362784 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 05:02:18.362869 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.359192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:41839 (local address 127.3.174.131:37309)
0808 05:02:18.359248 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:18.359262 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:18.359337 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:18.359528 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:18.359529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:18.359531 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:18.359604 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.359606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.360538 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.360538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.361870 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.361873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.362595 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.362596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.362657 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.362662 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:02:18.362716 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 05:02:18.362764 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":57}
I20260808 05:02:18.362867 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.359192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:36817 (local address 127.3.174.131:39741)
0808 05:02:18.359503 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:18.359513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:18.359561 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:18.359672 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:18.359673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:18.359674 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:18.359729 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.359731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.360538 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.360538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.361869 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:18.361873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.362579 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.362580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:18.362657 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.362662 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:02:18.362716 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 05:02:18.362762 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":177,"thread_start_us":111,"threads_started":1}
I20260808 05:02:18.363206 2501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77e426392cd0476ea66e5e1c11f39735" candidate_uuid: "bea503ffbbf44851a351354706b58eaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82877d98250246b4aa693c6aa0fc1390" is_pre_election: true
I20260808 05:02:18.363449 2501 raft_consensus.cc:2454] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bea503ffbbf44851a351354706b58eaa in term 0.
I20260808 05:02:18.363683 2860 leader_election.cc:302] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82877d98250246b4aa693c6aa0fc1390, bea503ffbbf44851a351354706b58eaa; no voters:
I20260808 05:02:18.363833 3086 raft_consensus.cc:2781] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:02:18.363816 2737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77e426392cd0476ea66e5e1c11f39735" candidate_uuid: "bea503ffbbf44851a351354706b58eaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "601fed68fee24fc0a4da4806b52ddde4" is_pre_election: true
I20260808 05:02:18.363886 3086 raft_consensus.cc:482] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:02:18.363902 3086 raft_consensus.cc:3037] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:18.363947 2737 raft_consensus.cc:2454] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bea503ffbbf44851a351354706b58eaa in term 0.
I20260808 05:02:18.364758 3086 raft_consensus.cc:504] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } }
I20260808 05:02:18.364885 3086 leader_election.cc:288] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [CANDIDATE]: Term 1 election: Requested vote from peers 601fed68fee24fc0a4da4806b52ddde4 (127.3.174.130:36817), 82877d98250246b4aa693c6aa0fc1390 (127.3.174.129:41839)
I20260808 05:02:18.365056 2737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77e426392cd0476ea66e5e1c11f39735" candidate_uuid: "bea503ffbbf44851a351354706b58eaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "601fed68fee24fc0a4da4806b52ddde4"
I20260808 05:02:18.365129 2737 raft_consensus.cc:3037] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:18.365372 2501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77e426392cd0476ea66e5e1c11f39735" candidate_uuid: "bea503ffbbf44851a351354706b58eaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82877d98250246b4aa693c6aa0fc1390"
I20260808 05:02:18.365477 2501 raft_consensus.cc:3037] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:18.365984 2737 raft_consensus.cc:2454] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bea503ffbbf44851a351354706b58eaa in term 1.
I20260808 05:02:18.366161 2863 leader_election.cc:302] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 601fed68fee24fc0a4da4806b52ddde4, bea503ffbbf44851a351354706b58eaa; no voters:
I20260808 05:02:18.366259 3086 raft_consensus.cc:2781] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:18.366254 2501 raft_consensus.cc:2454] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bea503ffbbf44851a351354706b58eaa in term 1.
I20260808 05:02:18.366422 3086 raft_consensus.cc:686] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 1 LEADER]: Becoming Leader. State: Replica: bea503ffbbf44851a351354706b58eaa, State: Running, Role: LEADER
I20260808 05:02:18.366540 3086 consensus_queue.cc:237] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } }
I20260808 05:02:18.367441 2252 catalog_manager.cc:5495] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa reported cstate change: term changed from 0 to 1, leader changed from <none> to bea503ffbbf44851a351354706b58eaa (127.3.174.131). New cstate: current_term: 1 leader_uuid: "bea503ffbbf44851a351354706b58eaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea503ffbbf44851a351354706b58eaa" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 34801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 } health_report { overall_health: UNKNOWN } } }
I20260808 05:02:18.377086 3067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.374794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51604 (local address 127.3.174.131:34801)
0808 05:02:18.374841 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:18.374843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:18.374852 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:18.374936 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:18.374937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:18.374956 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:18.375017 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:18.375130 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.375604 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.376368 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:18.376567 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:18.376620 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:18.376769 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:18.376870 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:18.376940 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:18.376977 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 05:02:18.376995 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 05:02:18.378913 2501 raft_consensus.cc:1261] T 77e426392cd0476ea66e5e1c11f39735 P 82877d98250246b4aa693c6aa0fc1390 [term 1 FOLLOWER]: Refusing update from remote peer bea503ffbbf44851a351354706b58eaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:18.378914 2737 raft_consensus.cc:1261] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [term 1 FOLLOWER]: Refusing update from remote peer bea503ffbbf44851a351354706b58eaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:18.379274 3086 consensus_queue.cc:1035] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "601fed68fee24fc0a4da4806b52ddde4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 36817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:18.379415 3090 consensus_queue.cc:1035] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "82877d98250246b4aa693c6aa0fc1390" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:18.382823 3098 mvcc.cc:205] Tried to move back new op lower bound from 7316133225997504512 to 7316133225949966336. Current Snapshot: MvccSnapshot[applied={T|T < 7316133225997504512 or (T in {7316133225997504512})}]
I20260808 05:02:18.383967 2251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:18.384066 2251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:18.384824 2251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56a95209a574496da22e51dc75527551: Sending DeleteTablet for 3 replicas of tablet 77e426392cd0476ea66e5e1c11f39735
I20260808 05:02:18.385130 2944 tablet_service.cc:1514] Processing DeleteTablet for tablet 77e426392cd0476ea66e5e1c11f39735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:18 UTC) from {username='slave'} at 127.0.0.1:51594
I20260808 05:02:18.385144 2701 tablet_service.cc:1514] Processing DeleteTablet for tablet 77e426392cd0476ea66e5e1c11f39735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:18 UTC) from {username='slave'} at 127.0.0.1:33462
I20260808 05:02:18.385197 2461 tablet_service.cc:1514] Processing DeleteTablet for tablet 77e426392cd0476ea66e5e1c11f39735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:18 UTC) from {username='slave'} at 127.0.0.1:44432
I20260808 05:02:18.385421 3112 tablet_replica.cc:330] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: stopping tablet replica
I20260808 05:02:18.385500 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 2355
I20260808 05:02:18.385520 3112 raft_consensus.cc:2229] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:02:18.385613 3112 raft_consensus.cc:2258] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:18.385740 3114 tablet_replica.cc:330] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: stopping tablet replica
I20260808 05:02:18.389312 3114 raft_consensus.cc:2229] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:02:18.389446 3114 raft_consensus.cc:2258] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:18.393572 3112 ts_tablet_manager.cc:1905] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:18.394081 3114 ts_tablet_manager.cc:1905] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:18.395329 3112 ts_tablet_manager.cc:1918] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:18.395395 3112 log.cc:1196] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/77e426392cd0476ea66e5e1c11f39735
I20260808 05:02:18.395624 3112 ts_tablet_manager.cc:1939] T 77e426392cd0476ea66e5e1c11f39735 P bea503ffbbf44851a351354706b58eaa: Deleting consensus metadata
I20260808 05:02:18.395643 3114 ts_tablet_manager.cc:1918] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:18.395725 3114 log.cc:1196] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/77e426392cd0476ea66e5e1c11f39735
I20260808 05:02:18.395982 3114 ts_tablet_manager.cc:1939] T 77e426392cd0476ea66e5e1c11f39735 P 601fed68fee24fc0a4da4806b52ddde4: Deleting consensus metadata
I20260808 05:02:18.396428 2230 catalog_manager.cc:4841] TS 601fed68fee24fc0a4da4806b52ddde4 (127.3.174.130:36817): tablet 77e426392cd0476ea66e5e1c11f39735 (table test-table [id=e9b28f2f2f6b4d7d84a6226a3f84a867]) successfully deleted
I20260808 05:02:18.397588 2229 catalog_manager.cc:4841] TS bea503ffbbf44851a351354706b58eaa (127.3.174.131:34801): tablet 77e426392cd0476ea66e5e1c11f39735 (table test-table [id=e9b28f2f2f6b4d7d84a6226a3f84a867]) successfully deleted
W20260808 05:02:18.398557 2229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:41839 (error 108)
I20260808 05:02:18.399053 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:18.398916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:41839 (local address 127.0.0.1:44434)
0808 05:02:18.398970 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:18.399017 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:41839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 05:02:18.399106 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 2580
W20260808 05:02:18.409564 2229 catalog_manager.cc:4584] TS 82877d98250246b4aa693c6aa0fc1390 (127.3.174.129:41839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77e426392cd0476ea66e5e1c11f39735: Network error: Client connection negotiation failed: client connection to 127.3.174.129:41839: connect: Connection refused (error 111)
I20260808 05:02:18.411216 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 2831
I20260808 05:02:18.432220 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 2201
2026-08-08T05:02:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (927 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:18 dist-test-slave-gjw7 krb5kdc[3123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:18 dist-test-slave-gjw7 krb5kdc[3123](info): set up 2 sockets
Aug 08 05:02:18 dist-test-slave-gjw7 krb5kdc[3123](info): commencing operation
krb5kdc: starting...
W20260808 05:02:19.637122 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:19 dist-test-slave-gjw7 krb5kdc[3123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:19.756337 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:37779
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:37779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:19.839949 3143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:19.840145 3143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:19.840166 3143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:19.841269 3143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:19.841325 3143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:19.841339 3143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:19.841351 3143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:19.841361 3143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:19.842801 3143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:37779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:37779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:19.842995 3143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:19.843183 3143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:19.847865 3155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:19.847918 3152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:19.848053 3143 server_base.cc:1034] running on GCE node
W20260808 05:02:19.847905 3151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:19.848421 3143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:19.848701 3143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:19.849840 3143 hybrid_clock.cc:648] HybridClock initialized: now 1786165339849825 us; error 33 us; skew 500 ppm
Aug 08 05:02:19 dist-test-slave-gjw7 krb5kdc[3123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165339, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:19.852255 3143 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:19.852680 3143 webserver.cc:460] Webserver started at http://127.3.174.190:40653/ using document root <none> and password file <none>
I20260808 05:02:19.852861 3143 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:19.852900 3143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:19.852989 3143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:19.854027 3143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "7bfacdaf0f34466995009602082007d0"
format_stamp: "Formatted at 2026-08-08 05:02:19 on dist-test-slave-gjw7"
server_key: "1da6a142592f0a63bd90f38d92a1bed9"
server_key_iv: "c3fc25018c78d9b429268a7dcae9b0c7"
server_key_version: "encryptionkey@0"
I20260808 05:02:19.854339 3143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "7bfacdaf0f34466995009602082007d0"
format_stamp: "Formatted at 2026-08-08 05:02:19 on dist-test-slave-gjw7"
server_key: "1da6a142592f0a63bd90f38d92a1bed9"
server_key_iv: "c3fc25018c78d9b429268a7dcae9b0c7"
server_key_version: "encryptionkey@0"
I20260808 05:02:19.855523 3143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 05:02:19.856179 3171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:19.856362 3143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:19.856446 3143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "7bfacdaf0f34466995009602082007d0"
format_stamp: "Formatted at 2026-08-08 05:02:19 on dist-test-slave-gjw7"
server_key: "1da6a142592f0a63bd90f38d92a1bed9"
server_key_iv: "c3fc25018c78d9b429268a7dcae9b0c7"
server_key_version: "encryptionkey@0"
I20260808 05:02:19.856514 3143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:19.880134 3143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:19.882983 3143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:19.883147 3143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:19.887684 3143 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:37779
I20260808 05:02:19.887723 3257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:37779 every 8 connection(s)
I20260808 05:02:19.888041 3143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:19.888913 3260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:19.891239 3260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0: Bootstrap starting.
I20260808 05:02:19.891597 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 3143
I20260808 05:02:19.891698 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:19.891872 3770 external_mini_cluster.cc:1428] Setting key 378c8b687305204997bad9a7b88b94f3
I20260808 05:02:19.891872 3260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:19.892124 3260 log.cc:824] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:19.892879 3260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0: No bootstrap required, opened a new log
I20260808 05:02:19.895264 3260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bfacdaf0f34466995009602082007d0" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37779 } }
I20260808 05:02:19.895438 3260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:19.895488 3260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bfacdaf0f34466995009602082007d0, State: Initialized, Role: FOLLOWER
I20260808 05:02:19.895623 3260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bfacdaf0f34466995009602082007d0" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37779 } }
I20260808 05:02:19.895697 3260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:19.895727 3260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:19.895774 3260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:19.896303 3260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bfacdaf0f34466995009602082007d0" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37779 } }
I20260808 05:02:19.896404 3260 leader_election.cc:302] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bfacdaf0f34466995009602082007d0; no voters:
I20260808 05:02:19.896574 3260 leader_election.cc:288] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:19.896625 3270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:19.896760 3270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [term 1 LEADER]: Becoming Leader. State: Replica: 7bfacdaf0f34466995009602082007d0, State: Running, Role: LEADER
I20260808 05:02:19.896822 3260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:19.896864 3270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bfacdaf0f34466995009602082007d0" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37779 } }
I20260808 05:02:19.897207 3270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bfacdaf0f34466995009602082007d0. Latest consensus state: current_term: 1 leader_uuid: "7bfacdaf0f34466995009602082007d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bfacdaf0f34466995009602082007d0" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37779 } } }
I20260808 05:02:19.897277 3270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:19.897697 3291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:19.898259 3291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:19.897681 3273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bfacdaf0f34466995009602082007d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bfacdaf0f34466995009602082007d0" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 37779 } } }
I20260808 05:02:19.899115 3273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0 [sys.catalog]: This master's current role is: LEADER
Aug 08 05:02:19 dist-test-slave-gjw7 krb5kdc[3123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:19.899827 3306 mvcc.cc:205] Tried to move back new op lower bound from 7316133232227258368 to 7316133232217939968. Current Snapshot: MvccSnapshot[applied={T|T < 7316133232227258368}]
I20260808 05:02:19.900267 3291 catalog_manager.cc:1347] Generated new cluster ID: a945bd3e1cae4f5f91a4a71cc1c74262
I20260808 05:02:19.900318 3291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:19.903838 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:19.892375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54488 (local address 127.3.174.190:37779)
0808 05:02:19.892532 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:19.892542 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:19.892614 (+ 72us) server_negotiation.cc:407] Connection header received
0808 05:02:19.893117 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:19.893124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:19.893173 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:19.893326 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:19.894116 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:19.895433 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:19.896296 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:19.896763 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:19.897209 (+ 446us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:19.899669 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:19.899674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:19.899674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:19.899686 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:19.901738 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:19.902145 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:19.902148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:19.902149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:19.902255 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:19.902476 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:19.902477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:19.902477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:19.902779 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:19.902936 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:19.903479 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 05:02:19.903621 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260808 05:02:19.931222 3291 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:19.931774 3291 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:19.939584 3291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0: Generated new TSK 0
I20260808 05:02:19.939788 3291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 05:02:19.975687 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37779
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:20.060591 3316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:20.060802 3316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:20.060834 3316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:20.061995 3316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:20.062032 3316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:20.062067 3316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:02:20.063504 3316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:20.063752 3316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:20.063956 3316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:20.068864 3325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.068876 3326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.068877 3333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:20.068993 3316 server_base.cc:1034] running on GCE node
I20260808 05:02:20.069200 3316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:20.069453 3316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:20.070598 3316 hybrid_clock.cc:648] HybridClock initialized: now 1786165340070581 us; error 33 us; skew 500 ppm
Aug 08 05:02:20 dist-test-slave-gjw7 krb5kdc[3123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165340, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:20.073067 3316 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:02:20.073521 3316 webserver.cc:460] Webserver started at http://127.3.174.129:40635/ using document root <none> and password file <none>
I20260808 05:02:20.073745 3316 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:20.073805 3316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:20.073936 3316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:20.074965 3316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "27f8cd8a03c4412db9024251018150bc"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "70228813823531f3917e0c9a9b557820"
server_key_iv: "b248b8aa7e21ca4948aa4dbe5d1a9654"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.075301 3316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "27f8cd8a03c4412db9024251018150bc"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "70228813823531f3917e0c9a9b557820"
server_key_iv: "b248b8aa7e21ca4948aa4dbe5d1a9654"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.076570 3316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:20.077241 3347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.077401 3316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:20.077486 3316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "27f8cd8a03c4412db9024251018150bc"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "70228813823531f3917e0c9a9b557820"
server_key_iv: "b248b8aa7e21ca4948aa4dbe5d1a9654"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.077553 3316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:20.086331 3316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:20.088095 3316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:20.088224 3316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:20.088421 3316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:20.088737 3316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:20.088773 3316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.088811 3316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:20.088835 3316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.094455 3316 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:33383
I20260808 05:02:20.094519 3557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:33383 every 8 connection(s)
I20260808 05:02:20.094807 3316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
Aug 08 05:02:20 dist-test-slave-gjw7 krb5kdc[3123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165340, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:20.101488 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 3316
I20260808 05:02:20.101588 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:02:20.101801 3770 external_mini_cluster.cc:1428] Setting key 5a08a239a81f1bd9bb5426b0b17f520a
I20260808 05:02:20.104108 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.095496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:45391 (local address 127.3.174.190:37779)
0808 05:02:20.095559 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.095561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.095719 (+ 158us) server_negotiation.cc:407] Connection header received
0808 05:02:20.096228 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.096229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.096239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.096320 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:20.097322 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.097911 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.098944 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.099198 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.099260 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.101361 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:20.101362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:20.101362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:20.101369 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:20.102839 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:20.103197 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:20.103198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:20.103199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:20.103262 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:20.103486 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:20.103487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:20.103487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:20.103593 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:20.103652 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.103915 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.104016 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 05:02:20.104390 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.095460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37779 (local address 127.3.174.129:45391)
0808 05:02:20.095608 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.095649 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:20.096148 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:20.096383 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:20.096386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:20.096965 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:20.097181 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.097186 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.097986 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.097986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.098859 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.098862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.099271 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.099272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.099398 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.099841 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:20.099849 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:20.101207 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:20.102910 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:20.102918 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:20.102920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:20.103130 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:20.103316 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:20.103317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:20.103317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:20.103429 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:20.103650 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:20.103657 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:20.103797 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 05:02:20.103884 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 05:02:20.104038 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260808 05:02:20.104838 3558 heartbeater.cc:347] Connected to a master server at 127.3.174.190:37779
I20260808 05:02:20.104930 3558 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:20.105108 3558 heartbeater.cc:511] Master 127.3.174.190:37779 requested a full tablet report, sending...
I20260808 05:02:20.105620 3196 ts_manager.cc:194] Registered new tserver with Master: 27f8cd8a03c4412db9024251018150bc (127.3.174.129:33383)
I20260808 05:02:20.106215 3196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:45391
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 05:02:20.125881 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37779
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:20.207543 3566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:20.207732 3566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:20.207753 3566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:20.208887 3566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:20.208930 3566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:20.208956 3566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:02:20.210635 3566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:20.210897 3566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:20.211160 3566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 05:02:20.216455 3566 server_base.cc:1034] running on GCE node
W20260808 05:02:20.216382 3573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.216367 3574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.216406 3579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:20.216764 3566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:20.217006 3566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:20.218149 3566 hybrid_clock.cc:648] HybridClock initialized: now 1786165340218138 us; error 27 us; skew 500 ppm
Aug 08 05:02:20 dist-test-slave-gjw7 krb5kdc[3123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165340, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:20.220566 3566 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:02:20.221002 3566 webserver.cc:460] Webserver started at http://127.3.174.130:44515/ using document root <none> and password file <none>
I20260808 05:02:20.221230 3566 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:20.221279 3566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:20.221433 3566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:20.222527 3566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "7389ced671ee497eb1c8784f0504e06d"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "1adafaa7b9bbc335a3b756973044aa13"
server_key_iv: "3645c37d68bdbb5ebb8baafcf34b2d61"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.222863 3566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "7389ced671ee497eb1c8784f0504e06d"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "1adafaa7b9bbc335a3b756973044aa13"
server_key_iv: "3645c37d68bdbb5ebb8baafcf34b2d61"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.224108 3566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:02:20.224766 3595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.224920 3566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:20.225004 3566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "7389ced671ee497eb1c8784f0504e06d"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "1adafaa7b9bbc335a3b756973044aa13"
server_key_iv: "3645c37d68bdbb5ebb8baafcf34b2d61"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.225070 3566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:20.235203 3566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:20.236905 3566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:20.237032 3566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:20.237232 3566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:20.237567 3566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:20.237601 3566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.237628 3566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:20.237645 3566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.243278 3566 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:39567
I20260808 05:02:20.243320 3782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:39567 every 8 connection(s)
I20260808 05:02:20.243616 3566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 05:02:20 dist-test-slave-gjw7 krb5kdc[3123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165340, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:20.250876 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 3566
I20260808 05:02:20.250986 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:02:20.251197 3770 external_mini_cluster.cc:1428] Setting key 30f0d08d9391e91f899d7cbd1a6e8039
I20260808 05:02:20.252740 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.244268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:52745 (local address 127.3.174.190:37779)
0808 05:02:20.244336 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.244339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.244560 (+ 221us) server_negotiation.cc:407] Connection header received
0808 05:02:20.245019 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.245020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.245030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.245081 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:20.246035 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.246542 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.247493 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.247676 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.247722 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.250003 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:20.250005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:20.250006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:20.250013 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:20.251343 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:20.251732 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:20.251733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:20.251734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:20.251794 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:20.252130 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:20.252131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:20.252131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:20.252244 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:20.252323 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.252567 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.252649 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 05:02:20.252918 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.244260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37779 (local address 127.3.174.130:52745)
0808 05:02:20.244429 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.244465 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:20.244939 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:20.245137 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:20.245141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:20.245756 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:20.245959 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.245964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.246629 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.246630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.247429 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.247432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.247769 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.247771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.247898 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.248435 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:20.248444 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:20.249910 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:20.251409 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:20.251417 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:20.251419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:20.251672 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:20.251960 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:20.251960 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:20.251961 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:20.252063 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:20.252303 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:20.252311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:20.252448 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 05:02:20.252538 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 05:02:20.252694 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260808 05:02:20.253391 3783 heartbeater.cc:347] Connected to a master server at 127.3.174.190:37779
I20260808 05:02:20.253485 3783 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:20.253675 3783 heartbeater.cc:511] Master 127.3.174.190:37779 requested a full tablet report, sending...
I20260808 05:02:20.254051 3196 ts_manager.cc:194] Registered new tserver with Master: 7389ced671ee497eb1c8784f0504e06d (127.3.174.130:39567)
I20260808 05:02:20.254415 3196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:52745
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 05:02:20.276026 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37779
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:20.358922 3795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:20.359115 3795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:20.359138 3795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:20.360356 3795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:20.360395 3795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:20.360420 3795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:02:20.361912 3795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:20.362099 3795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:20.362296 3795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:20.367046 3812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.367069 3807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.367318 3810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:20.367290 3795 server_base.cc:1034] running on GCE node
I20260808 05:02:20.367499 3795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:20.367692 3795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:20.368839 3795 hybrid_clock.cc:648] HybridClock initialized: now 1786165340368809 us; error 43 us; skew 500 ppm
Aug 08 05:02:20 dist-test-slave-gjw7 krb5kdc[3123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165340, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:20.371372 3795 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:02:20.371842 3795 webserver.cc:460] Webserver started at http://127.3.174.131:35553/ using document root <none> and password file <none>
I20260808 05:02:20.372117 3795 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:20.372167 3795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:20.372290 3795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:20.373359 3795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "045fa6c100d44385a1dd8d10c63575f1"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "ffbc66f68fb4cd17fe33592369135299"
server_key_iv: "cbd69d103ca985a5b820a1662312e39a"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.373689 3795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "045fa6c100d44385a1dd8d10c63575f1"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "ffbc66f68fb4cd17fe33592369135299"
server_key_iv: "cbd69d103ca985a5b820a1662312e39a"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.374933 3795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:02:20.375615 3822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.375787 3795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:20.375872 3795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "045fa6c100d44385a1dd8d10c63575f1"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "ffbc66f68fb4cd17fe33592369135299"
server_key_iv: "cbd69d103ca985a5b820a1662312e39a"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.375944 3795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:20.384802 3795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:20.386502 3795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:20.386628 3795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:20.386835 3795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:20.387116 3795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:20.387151 3795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.387177 3795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:20.387202 3795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.392866 3795 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:44831
I20260808 05:02:20.392928 4019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:44831 every 8 connection(s)
I20260808 05:02:20.393219 3795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
Aug 08 05:02:20 dist-test-slave-gjw7 krb5kdc[3123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165340, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:20.401465 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 3795
I20260808 05:02:20.401571 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 05:02:20.401731 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.393896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:51359 (local address 127.3.174.190:37779)
0808 05:02:20.393956 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.393959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.394140 (+ 181us) server_negotiation.cc:407] Connection header received
0808 05:02:20.394606 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.394607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.394617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.394661 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:20.395537 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.396038 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.397091 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.397281 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.397380 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.399477 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:20.399479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:20.399479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:20.399486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:20.400583 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:20.400931 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:20.400932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:20.400932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:20.400975 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:20.401147 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:20.401148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:20.401148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:20.401234 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:20.401277 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.401567 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.401641 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 05:02:20.401779 3770 external_mini_cluster.cc:1428] Setting key d5964cdca59ee73dd4197309433978b3
I20260808 05:02:20.401889 4033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.393877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:37779 (local address 127.3.174.131:51359)
0808 05:02:20.394032 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.394068 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:20.394531 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:20.394739 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:20.394742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:20.395285 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:20.395472 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.395477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.396125 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.396126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.397029 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.397031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.397400 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.397401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.397529 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.398001 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:20.398009 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:20.399389 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:20.400655 (+ 1266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:20.400662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:20.400664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:20.400872 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:20.401020 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:20.401020 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:20.401020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:20.401107 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:20.401305 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:20.401312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:20.401438 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 05:02:20.401527 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 05:02:20.401690 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260808 05:02:20.402287 4022 heartbeater.cc:347] Connected to a master server at 127.3.174.190:37779
I20260808 05:02:20.402392 4022 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:20.402590 4022 heartbeater.cc:511] Master 127.3.174.190:37779 requested a full tablet report, sending...
I20260808 05:02:20.402937 3194 ts_manager.cc:194] Registered new tserver with Master: 045fa6c100d44385a1dd8d10c63575f1 (127.3.174.131:44831)
I20260808 05:02:20.403266 3194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:51359
I20260808 05:02:20.413489 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:02:20.420408 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.414802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54500 (local address 127.3.174.190:37779)
0808 05:02:20.414854 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.414856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.414954 (+ 98us) server_negotiation.cc:407] Connection header received
0808 05:02:20.414999 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.414999 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.415007 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.415050 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:20.415558 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.416037 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.416721 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.416899 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.416943 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.418223 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:20.418224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:20.418224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:20.418230 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:20.419406 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:20.419751 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:20.419752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:20.419752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:20.419803 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:20.419982 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:20.419982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:20.419982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:20.420104 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:20.420182 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.420256 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.420321 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 05:02:20.422052 3194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:02:20.423601 3194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:02:20.430037 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.425739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33383 (local address 127.0.0.1:54300)
0808 05:02:20.426079 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.426116 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:20.426170 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:20.426516 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:20.426519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:20.426521 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:20.426592 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.426594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.427936 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.427937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.428849 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.428851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.429725 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.429726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.429809 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.429824 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 05:02:20.429901 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 05:02:20.429951 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":58,"thread_start_us":76,"threads_started":1}
I20260808 05:02:20.430294 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.425972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:44831 (local address 127.0.0.1:36846)
0808 05:02:20.426609 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.426619 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:20.426676 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:20.426845 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:20.426846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:20.426848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:20.426896 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.426898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.427822 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.427823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.429122 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.429125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.430052 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.430054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.430122 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.430126 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:20.430178 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 05:02:20.430215 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":31,"thread_start_us":39,"threads_started":1}
I20260808 05:02:20.430068 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.425883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54300 (local address 127.3.174.129:33383)
0808 05:02:20.426341 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.426345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.426357 (+ 12us) server_negotiation.cc:407] Connection header received
0808 05:02:20.426401 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.426403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.426419 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.426497 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:20.426676 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.427784 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.428934 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.429641 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.429716 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.429786 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.429938 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.429984 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":89,"threads_started":1}
I20260808 05:02:20.430565 4048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.426092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36846 (local address 127.3.174.131:44831)
0808 05:02:20.426442 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.426447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.426692 (+ 245us) server_negotiation.cc:407] Connection header received
0808 05:02:20.426756 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.426758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.426773 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.426842 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:20.426955 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.427707 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.429211 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.429906 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.429965 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.430018 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.430439 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.430486 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260808 05:02:20.430622 4047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.425932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.3.174.130:39567)
0808 05:02:20.426144 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.426150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.426284 (+ 134us) server_negotiation.cc:407] Connection header received
0808 05:02:20.426372 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.426375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.426393 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.426467 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:20.426672 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.427810 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.429550 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.430277 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.430347 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.430404 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.430497 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.430541 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":86,"threads_started":1}
I20260808 05:02:20.430789 4042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.425788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:39567 (local address 127.0.0.1:60496)
0808 05:02:20.426199 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.426209 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:20.426285 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:20.426552 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:20.426553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:20.426554 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:20.426593 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.426594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.427894 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.427895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.429464 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.429466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.430357 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.430358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.430419 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.430423 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:20.430482 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 05:02:20.430520 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":86,"thread_start_us":58,"threads_started":1}
I20260808 05:02:20.431195 3430 tablet_service.cc:1467] Processing CreateTablet for tablet b4112de2e69f474a98f6fada5b9f0f94 (DEFAULT_TABLE table=test-table [id=3b1103f2563a473c93f8e7ae516bf9a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:20.431195 3915 tablet_service.cc:1467] Processing CreateTablet for tablet b4112de2e69f474a98f6fada5b9f0f94 (DEFAULT_TABLE table=test-table [id=3b1103f2563a473c93f8e7ae516bf9a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:20.431403 3671 tablet_service.cc:1467] Processing CreateTablet for tablet b4112de2e69f474a98f6fada5b9f0f94 (DEFAULT_TABLE table=test-table [id=3b1103f2563a473c93f8e7ae516bf9a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:20.431550 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4112de2e69f474a98f6fada5b9f0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:20.431550 3915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4112de2e69f474a98f6fada5b9f0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:20.431689 3671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4112de2e69f474a98f6fada5b9f0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:20.434444 4050 tablet_bootstrap.cc:492] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: Bootstrap starting.
I20260808 05:02:20.435110 4050 tablet_bootstrap.cc:654] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:20.435144 4052 tablet_bootstrap.cc:492] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc: Bootstrap starting.
I20260808 05:02:20.435613 4050 log.cc:824] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:20.435637 4052 tablet_bootstrap.cc:654] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:20.436177 4052 log.cc:824] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:20.436427 4050 tablet_bootstrap.cc:492] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: No bootstrap required, opened a new log
I20260808 05:02:20.436493 4050 ts_tablet_manager.cc:1397] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:02:20.436488 4053 tablet_bootstrap.cc:492] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: Bootstrap starting.
I20260808 05:02:20.436802 4052 tablet_bootstrap.cc:492] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc: No bootstrap required, opened a new log
I20260808 05:02:20.436852 4052 ts_tablet_manager.cc:1397] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:02:20.437889 4053 tablet_bootstrap.cc:654] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:20.438236 4050 raft_consensus.cc:348] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } }
I20260808 05:02:20.438388 4050 raft_consensus.cc:374] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:20.438402 4053 log.cc:824] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:20.438426 4050 raft_consensus.cc:729] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7389ced671ee497eb1c8784f0504e06d, State: Initialized, Role: FOLLOWER
I20260808 05:02:20.438560 4050 consensus_queue.cc:260] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } }
I20260808 05:02:20.438737 4052 raft_consensus.cc:348] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } }
I20260808 05:02:20.438848 4050 ts_tablet_manager.cc:1428] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 05:02:20.438892 4052 raft_consensus.cc:374] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:20.438928 4052 raft_consensus.cc:729] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f8cd8a03c4412db9024251018150bc, State: Initialized, Role: FOLLOWER
I20260808 05:02:20.438974 3783 heartbeater.cc:503] Master 127.3.174.190:37779 was elected leader, sending a full tablet report...
I20260808 05:02:20.439035 4052 consensus_queue.cc:260] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } }
I20260808 05:02:20.439220 4053 tablet_bootstrap.cc:492] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: No bootstrap required, opened a new log
I20260808 05:02:20.439302 4053 ts_tablet_manager.cc:1397] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 05:02:20.439317 4052 ts_tablet_manager.cc:1428] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 05:02:20.439343 3558 heartbeater.cc:503] Master 127.3.174.190:37779 was elected leader, sending a full tablet report...
I20260808 05:02:20.441022 4053 raft_consensus.cc:348] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } }
I20260808 05:02:20.441162 4053 raft_consensus.cc:374] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:20.441200 4053 raft_consensus.cc:729] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 045fa6c100d44385a1dd8d10c63575f1, State: Initialized, Role: FOLLOWER
I20260808 05:02:20.441336 4053 consensus_queue.cc:260] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } }
I20260808 05:02:20.441633 4053 ts_tablet_manager.cc:1428] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 05:02:20.441721 4022 heartbeater.cc:503] Master 127.3.174.190:37779 was elected leader, sending a full tablet report...
W20260808 05:02:20.494181 3785 tablet.cc:2369] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:02:20.532780 4066 raft_consensus.cc:482] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:20.532946 4066 raft_consensus.cc:504] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } }
I20260808 05:02:20.533448 4066 leader_election.cc:288] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 045fa6c100d44385a1dd8d10c63575f1 (127.3.174.131:44831), 7389ced671ee497eb1c8784f0504e06d (127.3.174.130:39567)
I20260808 05:02:20.536870 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.533617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:39567 (local address 127.3.174.129:33791)
0808 05:02:20.533674 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.533686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:20.533731 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:20.533952 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:20.533954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:20.533957 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:20.534032 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.534034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.534742 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.534743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.535798 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.535801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.536611 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.536613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.536691 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.536695 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:20.536748 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 05:02:20.536795 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 05:02:20.536883 4047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.533764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:33791 (local address 127.3.174.130:39567)
0808 05:02:20.533814 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.533816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.533826 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:20.533860 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.533861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.533873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.533940 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:20.534111 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.534658 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.535920 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.536523 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.536602 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.536613 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.536771 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.536808 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 05:02:20.536897 4048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.533856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:39595 (local address 127.3.174.131:44831)
0808 05:02:20.533901 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.533903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.533951 (+ 48us) server_negotiation.cc:407] Connection header received
0808 05:02:20.534038 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.534040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.534049 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.534092 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:20.534233 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.534976 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.536160 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.536553 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.536615 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.536624 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.536802 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.536842 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 05:02:20.537129 4073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.533617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:44831 (local address 127.3.174.129:39595)
0808 05:02:20.533897 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.533907 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:20.533973 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:20.534117 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:20.534117 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:20.534119 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:20.534173 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.534174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.535073 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.535074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.536079 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.536082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.536652 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.536653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.536717 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.536720 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:02:20.536999 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 05:02:20.537055 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":14,"thread_start_us":119,"threads_started":1}
I20260808 05:02:20.537442 3947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4112de2e69f474a98f6fada5b9f0f94" candidate_uuid: "27f8cd8a03c4412db9024251018150bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "045fa6c100d44385a1dd8d10c63575f1" is_pre_election: true
I20260808 05:02:20.537611 3947 raft_consensus.cc:2454] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27f8cd8a03c4412db9024251018150bc in term 0.
I20260808 05:02:20.537644 3703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4112de2e69f474a98f6fada5b9f0f94" candidate_uuid: "27f8cd8a03c4412db9024251018150bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7389ced671ee497eb1c8784f0504e06d" is_pre_election: true
I20260808 05:02:20.537801 3703 raft_consensus.cc:2454] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27f8cd8a03c4412db9024251018150bc in term 0.
I20260808 05:02:20.537853 3349 leader_election.cc:302] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 045fa6c100d44385a1dd8d10c63575f1, 27f8cd8a03c4412db9024251018150bc; no voters:
I20260808 05:02:20.537989 4066 raft_consensus.cc:2781] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:02:20.538038 4066 raft_consensus.cc:482] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:02:20.538051 4066 raft_consensus.cc:3037] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:20.538774 4066 raft_consensus.cc:504] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } }
I20260808 05:02:20.538902 4066 leader_election.cc:288] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [CANDIDATE]: Term 1 election: Requested vote from peers 045fa6c100d44385a1dd8d10c63575f1 (127.3.174.131:44831), 7389ced671ee497eb1c8784f0504e06d (127.3.174.130:39567)
I20260808 05:02:20.539059 3947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4112de2e69f474a98f6fada5b9f0f94" candidate_uuid: "27f8cd8a03c4412db9024251018150bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "045fa6c100d44385a1dd8d10c63575f1"
I20260808 05:02:20.539106 3703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4112de2e69f474a98f6fada5b9f0f94" candidate_uuid: "27f8cd8a03c4412db9024251018150bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7389ced671ee497eb1c8784f0504e06d"
I20260808 05:02:20.539155 3947 raft_consensus.cc:3037] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:20.539183 3703 raft_consensus.cc:3037] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:20.539930 3703 raft_consensus.cc:2454] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27f8cd8a03c4412db9024251018150bc in term 1.
I20260808 05:02:20.540091 3947 raft_consensus.cc:2454] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27f8cd8a03c4412db9024251018150bc in term 1.
I20260808 05:02:20.540115 3350 leader_election.cc:302] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27f8cd8a03c4412db9024251018150bc, 7389ced671ee497eb1c8784f0504e06d; no voters:
I20260808 05:02:20.540189 4066 raft_consensus.cc:2781] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:20.540354 4066 raft_consensus.cc:686] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [term 1 LEADER]: Becoming Leader. State: Replica: 27f8cd8a03c4412db9024251018150bc, State: Running, Role: LEADER
I20260808 05:02:20.540458 4066 consensus_queue.cc:237] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } }
I20260808 05:02:20.541321 3194 catalog_manager.cc:5495] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f8cd8a03c4412db9024251018150bc (127.3.174.129). New cstate: current_term: 1 leader_uuid: "27f8cd8a03c4412db9024251018150bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f8cd8a03c4412db9024251018150bc" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 33383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 } health_report { overall_health: UNKNOWN } } }
I20260808 05:02:20.573992 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.571819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54304 (local address 127.3.174.129:33383)
0808 05:02:20.571872 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.571874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.571883 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:20.571943 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.571944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.571963 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.572027 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:20.572139 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.572610 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.573319 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.573522 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.573577 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.573697 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:20.573787 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:20.573858 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.573878 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.573888 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 05:02:20.575780 3947 raft_consensus.cc:1261] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [term 1 FOLLOWER]: Refusing update from remote peer 27f8cd8a03c4412db9024251018150bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:20.575780 3703 raft_consensus.cc:1261] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [term 1 FOLLOWER]: Refusing update from remote peer 27f8cd8a03c4412db9024251018150bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:20.576119 4066 consensus_queue.cc:1035] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "045fa6c100d44385a1dd8d10c63575f1" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 44831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:20.576260 4074 consensus_queue.cc:1035] T b4112de2e69f474a98f6fada5b9f0f94 P 27f8cd8a03c4412db9024251018150bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7389ced671ee497eb1c8784f0504e06d" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 39567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:20.578797 4083 mvcc.cc:205] Tried to move back new op lower bound from 7316133234995789824 to 7316133234854322176. Current Snapshot: MvccSnapshot[applied={T|T < 7316133234995789824}]
I20260808 05:02:20.579221 4081 mvcc.cc:205] Tried to move back new op lower bound from 7316133234995789824 to 7316133234854322176. Current Snapshot: MvccSnapshot[applied={T|T < 7316133234995789824}]
I20260808 05:02:20.581012 3196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:20.581103 3196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:20.581879 3196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7bfacdaf0f34466995009602082007d0: Sending DeleteTablet for 3 replicas of tablet b4112de2e69f474a98f6fada5b9f0f94
I20260808 05:02:20.582228 3671 tablet_service.cc:1514] Processing DeleteTablet for tablet b4112de2e69f474a98f6fada5b9f0f94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:20 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:60496
I20260808 05:02:20.582247 3915 tablet_service.cc:1514] Processing DeleteTablet for tablet b4112de2e69f474a98f6fada5b9f0f94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:20 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:36846
I20260808 05:02:20.582443 4096 tablet_replica.cc:330] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: stopping tablet replica
I20260808 05:02:20.582561 4099 tablet_replica.cc:330] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: stopping tablet replica
I20260808 05:02:20.582571 4096 raft_consensus.cc:2229] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:02:20.582700 4099 raft_consensus.cc:2229] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:02:20.582700 3430 tablet_service.cc:1514] Processing DeleteTablet for tablet b4112de2e69f474a98f6fada5b9f0f94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:20 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:54300
I20260808 05:02:20.582726 4096 raft_consensus.cc:2258] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:20.582859 4099 raft_consensus.cc:2258] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:20.582893 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 3316
I20260808 05:02:20.583627 4099 ts_tablet_manager.cc:1905] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:20.585053 4099 ts_tablet_manager.cc:1918] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:20.585129 4099 log.cc:1196] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/b4112de2e69f474a98f6fada5b9f0f94
I20260808 05:02:20.585395 4099 ts_tablet_manager.cc:1939] T b4112de2e69f474a98f6fada5b9f0f94 P 045fa6c100d44385a1dd8d10c63575f1: Deleting consensus metadata
I20260808 05:02:20.585757 3173 catalog_manager.cc:4841] TS 045fa6c100d44385a1dd8d10c63575f1 (127.3.174.131:44831): tablet b4112de2e69f474a98f6fada5b9f0f94 (table test-table [id=3b1103f2563a473c93f8e7ae516bf9a5]) successfully deleted
I20260808 05:02:20.590833 4096 ts_tablet_manager.cc:1905] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:20.592226 4096 ts_tablet_manager.cc:1918] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:20.592288 4096 log.cc:1196] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/b4112de2e69f474a98f6fada5b9f0f94
I20260808 05:02:20.592510 4096 ts_tablet_manager.cc:1939] T b4112de2e69f474a98f6fada5b9f0f94 P 7389ced671ee497eb1c8784f0504e06d: Deleting consensus metadata
I20260808 05:02:20.592888 3176 catalog_manager.cc:4841] TS 7389ced671ee497eb1c8784f0504e06d (127.3.174.130:39567): tablet b4112de2e69f474a98f6fada5b9f0f94 (table test-table [id=3b1103f2563a473c93f8e7ae516bf9a5]) successfully deleted
W20260808 05:02:20.595317 3173 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:33383 (error 108)
I20260808 05:02:20.595782 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.595652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33383 (local address 127.0.0.1:54316)
0808 05:02:20.595707 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.595750 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 05:02:20.595860 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 3566
W20260808 05:02:20.595930 3173 catalog_manager.cc:4584] TS 27f8cd8a03c4412db9024251018150bc (127.3.174.129:33383): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4112de2e69f474a98f6fada5b9f0f94: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33383: connect: Connection refused (error 111)
I20260808 05:02:20.634419 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 3795
I20260808 05:02:20.648545 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.648365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33383 (local address 127.0.0.1:54320)
0808 05:02:20.648447 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.648495 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 05:02:20.648891 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.648801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:33383 (local address 127.0.0.1:54334)
0808 05:02:20.648840 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.648867 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:33383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 05:02:20.651319 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 3143
2026-08-08T05:02:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2219 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T05:02:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:20Z Disabled control of system clock
I20260808 05:02:20.684784 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:36003
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:36003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:20.768114 4107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:20.768353 4107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:20.768374 4107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:20.769618 4107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:20.769665 4107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:20.769677 4107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:20.769691 4107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:20.769714 4107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:20.771085 4107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:36003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:36003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:20.771345 4107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:20.771641 4107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:20.776357 4116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.776368 4123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.776404 4118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:20.776496 4107 server_base.cc:1034] running on GCE node
I20260808 05:02:20.776830 4107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:20.777076 4107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:20.778206 4107 hybrid_clock.cc:648] HybridClock initialized: now 1786165340778189 us; error 33 us; skew 500 ppm
I20260808 05:02:20.783321 4107 webserver.cc:460] Webserver started at http://127.3.174.190:41725/ using document root <none> and password file <none>
I20260808 05:02:20.783569 4107 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:20.783624 4107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:20.783756 4107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:20.784847 4107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "1c61f19196734ca5b49b1917c9900b1c"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "22dfd8b419a21bd04806e549d3796c73"
server_key_iv: "a52875f8b7d13252460b36229409fd3b"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.785189 4107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "1c61f19196734ca5b49b1917c9900b1c"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "22dfd8b419a21bd04806e549d3796c73"
server_key_iv: "a52875f8b7d13252460b36229409fd3b"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.786470 4107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:02:20.787154 4133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.787307 4107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:20.787395 4107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "1c61f19196734ca5b49b1917c9900b1c"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "22dfd8b419a21bd04806e549d3796c73"
server_key_iv: "a52875f8b7d13252460b36229409fd3b"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.787459 4107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:20.809679 4107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:20.810048 4107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:20.810206 4107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:20.814302 4107 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:36003
I20260808 05:02:20.814347 4228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:36003 every 8 connection(s)
I20260808 05:02:20.814697 4107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:20.815672 4230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:20.818148 4230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c: Bootstrap starting.
I20260808 05:02:20.818702 4230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:20.818985 4230 log.cc:824] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:20.819144 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 4107
I20260808 05:02:20.819249 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:20.819427 3770 external_mini_cluster.cc:1428] Setting key 08f5f29e338831fa622ccf63f9534659
I20260808 05:02:20.819777 4230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c: No bootstrap required, opened a new log
I20260808 05:02:20.821832 4230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c61f19196734ca5b49b1917c9900b1c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36003 } }
I20260808 05:02:20.821973 4230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:20.822005 4230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c61f19196734ca5b49b1917c9900b1c, State: Initialized, Role: FOLLOWER
I20260808 05:02:20.822134 4230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c61f19196734ca5b49b1917c9900b1c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36003 } }
I20260808 05:02:20.822206 4230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:20.822247 4230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:20.822294 4230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:20.822842 4230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c61f19196734ca5b49b1917c9900b1c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36003 } }
I20260808 05:02:20.822943 4230 leader_election.cc:302] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c61f19196734ca5b49b1917c9900b1c; no voters:
I20260808 05:02:20.823108 4230 leader_election.cc:288] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:20.823150 4237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:20.823256 4237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [term 1 LEADER]: Becoming Leader. State: Replica: 1c61f19196734ca5b49b1917c9900b1c, State: Running, Role: LEADER
I20260808 05:02:20.823379 4230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:20.823402 4237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c61f19196734ca5b49b1917c9900b1c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36003 } }
I20260808 05:02:20.823853 4239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c61f19196734ca5b49b1917c9900b1c. Latest consensus state: current_term: 1 leader_uuid: "1c61f19196734ca5b49b1917c9900b1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c61f19196734ca5b49b1917c9900b1c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36003 } } }
I20260808 05:02:20.823858 4238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c61f19196734ca5b49b1917c9900b1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c61f19196734ca5b49b1917c9900b1c" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36003 } } }
I20260808 05:02:20.823948 4239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:20.823968 4238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:20.824244 4246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:20.824770 4246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:20.826364 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.819935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.3.174.190:36003)
0808 05:02:20.820132 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.820140 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.820166 (+ 26us) server_negotiation.cc:407] Connection header received
0808 05:02:20.821003 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.821012 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.821076 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.821190 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:20.821497 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.822857 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.823887 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.824475 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.824569 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.825593 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:20.825598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:20.825599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:20.825811 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:20.825855 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:20.825871 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:20.825933 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.826054 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.826142 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260808 05:02:20.826536 4246 catalog_manager.cc:1347] Generated new cluster ID: 21a276f76d59473a8ee608b271930ce2
I20260808 05:02:20.826581 4246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:20.836077 4246 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:20.836575 4246 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:20.842119 4246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c: Generated new TSK 0
I20260808 05:02:20.842310 4246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:20.844422 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36003
--builtin_ntp_servers=127.3.174.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:20.932119 4266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:20.932319 4266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:20.932340 4266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:20.933547 4266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:20.933589 4266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:20.933614 4266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:02:20.935016 4266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:20.935261 4266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:20.935562 4266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:20.940623 4277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.940632 4280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:20.940641 4273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:20.941483 4266 server_base.cc:1034] running on GCE node
I20260808 05:02:20.941733 4266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:20.942022 4266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:20.943189 4266 hybrid_clock.cc:648] HybridClock initialized: now 1786165340943175 us; error 29 us; skew 500 ppm
I20260808 05:02:20.944800 4266 webserver.cc:460] Webserver started at http://127.3.174.129:38519/ using document root <none> and password file <none>
I20260808 05:02:20.945082 4266 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:20.945150 4266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:20.945324 4266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:20.946744 4266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "2f8efd00202c4b2181f1acd4cbabd8c8"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "06b521a63ce2ea8db1d059ae795dc277"
server_key_iv: "80b4ec187991f54a59136832fa7afa19"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.947144 4266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "2f8efd00202c4b2181f1acd4cbabd8c8"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "06b521a63ce2ea8db1d059ae795dc277"
server_key_iv: "80b4ec187991f54a59136832fa7afa19"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.948740 4266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:20.949729 4288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.949922 4266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:20.950021 4266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "2f8efd00202c4b2181f1acd4cbabd8c8"
format_stamp: "Formatted at 2026-08-08 05:02:20 on dist-test-slave-gjw7"
server_key: "06b521a63ce2ea8db1d059ae795dc277"
server_key_iv: "80b4ec187991f54a59136832fa7afa19"
server_key_version: "encryptionkey@0"
I20260808 05:02:20.950104 4266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:20.962476 4266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:20.962819 4266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:20.962949 4266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:20.963157 4266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:20.963478 4266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:20.963510 4266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.963533 4266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:20.963549 4266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:20.969444 4266 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:39597
I20260808 05:02:20.969494 4497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:39597 every 8 connection(s)
I20260808 05:02:20.969825 4266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:02:20.972615 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 4266
I20260808 05:02:20.972719 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:02:20.972906 3770 external_mini_cluster.cc:1428] Setting key 2c9f0b8c16c8c0a79bfa73845377e85d
I20260808 05:02:20.973892 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36003
--builtin_ntp_servers=127.3.174.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 05:02:20.975442 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.970648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:47095 (local address 127.3.174.190:36003)
0808 05:02:20.970721 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:20.970723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:20.970936 (+ 213us) server_negotiation.cc:407] Connection header received
0808 05:02:20.971436 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:20.971437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:20.971451 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:20.971510 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:20.972156 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.972669 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.973934 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.974212 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.974284 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.975152 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:20.975154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:20.975154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:20.975159 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:20.975184 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:20.975188 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:20.975231 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:20.975325 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 05:02:20.975360 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 05:02:20.975646 4502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:20.970613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:36003 (local address 127.3.174.129:47095)
0808 05:02:20.970814 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:20.970855 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:20.971361 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:20.971582 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:20.971586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:20.971595 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:20.972094 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.972098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.972780 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.972782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.973850 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:20.973854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:20.974332 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:20.974334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:20.974461 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:20.974996 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 05:02:20.974996 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:20.975030 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 05:02:20.975056 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:20.975240 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:20.975246 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:20.975250 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:20.975323 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 05:02:20.975437 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260808 05:02:20.976156 4500 heartbeater.cc:347] Connected to a master server at 127.3.174.190:36003
I20260808 05:02:20.976277 4500 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:20.976490 4500 heartbeater.cc:511] Master 127.3.174.190:36003 requested a full tablet report, sending...
I20260808 05:02:20.977065 4160 ts_manager.cc:194] Registered new tserver with Master: 2f8efd00202c4b2181f1acd4cbabd8c8 (127.3.174.129:39597)
I20260808 05:02:20.977771 4160 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.129:47095
W20260808 05:02:21.060362 4503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:21.060580 4503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:21.060608 4503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:21.061772 4503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:21.061812 4503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:21.061846 4503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:02:21.063223 4503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:21.063467 4503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:21.063738 4503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:21.068722 4511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:21.068729 4517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:21.068827 4503 server_base.cc:1034] running on GCE node
W20260808 05:02:21.068773 4513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:21.069120 4503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:21.069410 4503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:21.070554 4503 hybrid_clock.cc:648] HybridClock initialized: now 1786165341070537 us; error 32 us; skew 500 ppm
I20260808 05:02:21.071756 4503 webserver.cc:460] Webserver started at http://127.3.174.130:38651/ using document root <none> and password file <none>
I20260808 05:02:21.071981 4503 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:21.072026 4503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:21.072157 4503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:21.073271 4503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "baf65160af524598aa8729c3918644b4"
format_stamp: "Formatted at 2026-08-08 05:02:21 on dist-test-slave-gjw7"
server_key: "9f498b10246de420c9b1b433f12f944a"
server_key_iv: "b924e11a1d6dd0e9fde75c5fcecb68eb"
server_key_version: "encryptionkey@0"
I20260808 05:02:21.073668 4503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "baf65160af524598aa8729c3918644b4"
format_stamp: "Formatted at 2026-08-08 05:02:21 on dist-test-slave-gjw7"
server_key: "9f498b10246de420c9b1b433f12f944a"
server_key_iv: "b924e11a1d6dd0e9fde75c5fcecb68eb"
server_key_version: "encryptionkey@0"
I20260808 05:02:21.074976 4503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:21.075659 4528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:21.075802 4503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:21.075888 4503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "baf65160af524598aa8729c3918644b4"
format_stamp: "Formatted at 2026-08-08 05:02:21 on dist-test-slave-gjw7"
server_key: "9f498b10246de420c9b1b433f12f944a"
server_key_iv: "b924e11a1d6dd0e9fde75c5fcecb68eb"
server_key_version: "encryptionkey@0"
I20260808 05:02:21.075965 4503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:21.093971 4503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:21.094283 4503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:21.094410 4503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:21.094622 4503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:21.094969 4503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:21.095006 4503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:21.095041 4503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:21.095058 4503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:21.100857 4503 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:37485
I20260808 05:02:21.100929 4728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:37485 every 8 connection(s)
I20260808 05:02:21.101212 4503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 05:02:21.106033 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.101974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:39175 (local address 127.3.174.190:36003)
0808 05:02:21.102038 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:21.102040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:21.102236 (+ 196us) server_negotiation.cc:407] Connection header received
0808 05:02:21.102776 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:21.102778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:21.102794 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:21.102846 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:21.103491 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.103983 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.104886 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.105060 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.105100 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.105835 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:21.105837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:21.105837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:21.105841 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:21.105855 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:21.105859 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:21.105896 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:21.105952 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 05:02:21.105974 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 05:02:21.106215 4734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.101938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:36003 (local address 127.3.174.130:39175)
0808 05:02:21.102122 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:21.102162 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:21.102691 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:21.102909 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:21.102913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:21.102921 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:21.103430 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.103435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.104045 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.104046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.104826 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.104829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.105111 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.105111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.105215 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.105710 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 05:02:21.105710 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:21.105738 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 05:02:21.105763 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:21.105898 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:21.105901 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:21.105903 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 05:02:21.105947 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 05:02:21.106022 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260808 05:02:21.106683 4729 heartbeater.cc:347] Connected to a master server at 127.3.174.190:36003
I20260808 05:02:21.106768 4729 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:21.106931 4729 heartbeater.cc:511] Master 127.3.174.190:36003 requested a full tablet report, sending...
I20260808 05:02:21.107250 4160 ts_manager.cc:194] Registered new tserver with Master: baf65160af524598aa8729c3918644b4 (127.3.174.130:37485)
I20260808 05:02:21.107618 4160 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.130:39175
I20260808 05:02:21.110500 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 4503
I20260808 05:02:21.110595 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:02:21.110749 3770 external_mini_cluster.cc:1428] Setting key b563a13a0e47ce0ae39b9e19db05be60
I20260808 05:02:21.111603 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36003
--builtin_ntp_servers=127.3.174.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:21.194877 4736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:21.195070 4736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:21.195097 4736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:21.196223 4736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:21.196264 4736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:21.196295 4736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:02:21.197669 4736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:36003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:21.197904 4736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:21.198172 4736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:21.202885 4744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:21.202921 4750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:21.202941 4745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:21.203233 4736 server_base.cc:1034] running on GCE node
I20260808 05:02:21.203447 4736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:21.203667 4736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:21.204804 4736 hybrid_clock.cc:648] HybridClock initialized: now 1786165341204791 us; error 28 us; skew 500 ppm
I20260808 05:02:21.206051 4736 webserver.cc:460] Webserver started at http://127.3.174.131:36607/ using document root <none> and password file <none>
I20260808 05:02:21.206295 4736 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:21.206354 4736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:21.206478 4736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:21.207727 4736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "5367bd8d26d6475fb86a67591cd7e759"
format_stamp: "Formatted at 2026-08-08 05:02:21 on dist-test-slave-gjw7"
server_key: "877e1f0695a4cd0e502dec57969be910"
server_key_iv: "abeba101cc088ef9106b6dfeca6153d7"
server_key_version: "encryptionkey@0"
I20260808 05:02:21.208060 4736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "5367bd8d26d6475fb86a67591cd7e759"
format_stamp: "Formatted at 2026-08-08 05:02:21 on dist-test-slave-gjw7"
server_key: "877e1f0695a4cd0e502dec57969be910"
server_key_iv: "abeba101cc088ef9106b6dfeca6153d7"
server_key_version: "encryptionkey@0"
I20260808 05:02:21.209494 4736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:21.210150 4761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:21.210304 4736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:21.210388 4736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "5367bd8d26d6475fb86a67591cd7e759"
format_stamp: "Formatted at 2026-08-08 05:02:21 on dist-test-slave-gjw7"
server_key: "877e1f0695a4cd0e502dec57969be910"
server_key_iv: "abeba101cc088ef9106b6dfeca6153d7"
server_key_version: "encryptionkey@0"
I20260808 05:02:21.210444 4736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:21.236373 4736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:21.236678 4736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:21.236784 4736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:21.237035 4736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:21.237396 4736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:21.237434 4736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:21.237460 4736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:21.237475 4736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:21.242874 4736 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:37651
I20260808 05:02:21.242951 4976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:37651 every 8 connection(s)
I20260808 05:02:21.243243 4736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:02:21.247213 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 4736
I20260808 05:02:21.247308 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 05:02:21.247463 3770 external_mini_cluster.cc:1428] Setting key ad54352cbf8ee7247a07c67dbcb1c33a
I20260808 05:02:21.248238 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.243942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:56515 (local address 127.3.174.190:36003)
0808 05:02:21.244007 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:21.244009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:21.244227 (+ 218us) server_negotiation.cc:407] Connection header received
0808 05:02:21.244724 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:21.244726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:21.244736 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:21.244774 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:21.245427 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.245906 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.246805 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.246981 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.247051 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.247941 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:21.247942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:21.247943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:21.247948 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:21.247975 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:21.247979 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:21.248035 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:21.248128 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 05:02:21.248156 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 05:02:21.248443 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.243927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:36003 (local address 127.3.174.131:56515)
0808 05:02:21.244113 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:21.244149 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:21.244633 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:21.244835 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:21.244838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:21.244847 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:21.245363 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.245368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.245972 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.245972 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.246736 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.246738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.247053 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.247053 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.247157 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.247784 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 05:02:21.247785 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:21.247824 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 05:02:21.247854 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:21.248044 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:21.248050 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:21.248053 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:02:21.248121 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 05:02:21.248223 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260808 05:02:21.248864 4977 heartbeater.cc:347] Connected to a master server at 127.3.174.190:36003
I20260808 05:02:21.248956 4977 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:21.249140 4977 heartbeater.cc:511] Master 127.3.174.190:36003 requested a full tablet report, sending...
I20260808 05:02:21.249511 4160 ts_manager.cc:194] Registered new tserver with Master: 5367bd8d26d6475fb86a67591cd7e759 (127.3.174.131:37651)
I20260808 05:02:21.249882 4160 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.131:56515
I20260808 05:02:21.259244 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:02:21.263465 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.260597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.3.174.190:36003)
0808 05:02:21.260653 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:21.260655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:21.260749 (+ 94us) server_negotiation.cc:407] Connection header received
0808 05:02:21.260812 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:21.260813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:21.260823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:21.260870 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:21.261020 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.261565 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.262293 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.262473 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.262527 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.263182 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:21.263183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:21.263183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:21.263188 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:21.263203 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:21.263207 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:21.263293 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:21.263368 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 05:02:21.263379 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 05:02:21.264837 4160 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:02:21.266489 4160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:02:21.272238 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.268716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59136 (local address 127.3.174.131:37651)
0808 05:02:21.268895 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:21.268899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:21.268908 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:21.268948 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:21.268950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:21.268962 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:21.269031 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:21.269208 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.270098 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.271309 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.271867 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.271931 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.272065 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:21.272116 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 05:02:21.272153 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260808 05:02:21.272228 4992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.268450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:37651 (local address 127.0.0.1:59136)
0808 05:02:21.268615 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:21.268644 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:21.268709 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:21.269053 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:21.269058 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:21.269060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:21.269125 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.269127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.270175 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.270176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.271141 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.271169 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.271946 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.271947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.272029 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.272043 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 05:02:21.272105 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 05:02:21.272160 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":55,"threads_started":1}
I20260808 05:02:21.272770 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.268450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:37485 (local address 127.0.0.1:39596)
0808 05:02:21.268628 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:21.268657 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:21.268711 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:21.269257 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:21.269259 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:21.269261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:21.269335 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.269337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.270720 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.270721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.271670 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.271673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.272537 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.272538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.272601 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.272605 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:21.272665 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 05:02:21.272705 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
I20260808 05:02:21.272799 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.268911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.3.174.130:37485)
0808 05:02:21.269089 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:21.269093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:21.269102 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:21.269137 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:21.269139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:21.269155 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:21.269406 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:21.269462 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.270643 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.271741 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.272458 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.272529 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.272608 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:21.272678 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 05:02:21.272709 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260808 05:02:21.273267 4848 tablet_service.cc:1467] Processing CreateTablet for tablet 4ad84ddb2b774fa093eb2af499a6917c (DEFAULT_TABLE table=test-table [id=1dd9731122d9494a894d2fc70eab76b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:21.273717 4848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ad84ddb2b774fa093eb2af499a6917c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:21.273904 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.268719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.3.174.129:39597)
0808 05:02:21.268904 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:21.268908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:21.269162 (+ 254us) server_negotiation.cc:407] Connection header received
0808 05:02:21.269209 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:21.269211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:21.269225 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:21.269323 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:21.269499 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.270634 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.272875 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.273587 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.273649 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.273731 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:21.273789 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 05:02:21.273827 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260808 05:02:21.273921 4993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.268452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39597 (local address 127.0.0.1:52492)
0808 05:02:21.269082 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:21.269092 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:21.269151 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:21.269392 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:21.269392 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:21.269394 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:21.269427 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.269428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.271748 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.271748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.272782 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.272785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.273662 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.273663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.273723 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.273727 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:21.273821 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 05:02:21.273862 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260808 05:02:21.274487 4379 tablet_service.cc:1467] Processing CreateTablet for tablet 4ad84ddb2b774fa093eb2af499a6917c (DEFAULT_TABLE table=test-table [id=1dd9731122d9494a894d2fc70eab76b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:21.274772 4379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ad84ddb2b774fa093eb2af499a6917c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:21.276393 5002 tablet_bootstrap.cc:492] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: Bootstrap starting.
I20260808 05:02:21.277052 5002 tablet_bootstrap.cc:654] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:21.276993 4620 tablet_service.cc:1467] Processing CreateTablet for tablet 4ad84ddb2b774fa093eb2af499a6917c (DEFAULT_TABLE table=test-table [id=1dd9731122d9494a894d2fc70eab76b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:21.277362 4620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ad84ddb2b774fa093eb2af499a6917c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:21.277537 5002 log.cc:824] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:21.278290 5002 tablet_bootstrap.cc:492] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: No bootstrap required, opened a new log
I20260808 05:02:21.278363 5002 ts_tablet_manager.cc:1397] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:02:21.280580 5008 tablet_bootstrap.cc:492] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: Bootstrap starting.
I20260808 05:02:21.280789 5004 tablet_bootstrap.cc:492] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8: Bootstrap starting.
I20260808 05:02:21.280918 5002 raft_consensus.cc:348] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } }
I20260808 05:02:21.281091 5002 raft_consensus.cc:374] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:21.281136 5002 raft_consensus.cc:729] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5367bd8d26d6475fb86a67591cd7e759, State: Initialized, Role: FOLLOWER
I20260808 05:02:21.281257 5008 tablet_bootstrap.cc:654] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:21.281261 5002 consensus_queue.cc:260] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } }
I20260808 05:02:21.281457 5004 tablet_bootstrap.cc:654] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:21.281569 5002 ts_tablet_manager.cc:1428] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 05:02:21.281625 5008 log.cc:824] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:21.281644 4977 heartbeater.cc:503] Master 127.3.174.190:36003 was elected leader, sending a full tablet report...
I20260808 05:02:21.282059 5004 log.cc:824] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:21.282524 5008 tablet_bootstrap.cc:492] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: No bootstrap required, opened a new log
I20260808 05:02:21.282598 5008 ts_tablet_manager.cc:1397] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:02:21.282862 5004 tablet_bootstrap.cc:492] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8: No bootstrap required, opened a new log
I20260808 05:02:21.282935 5004 ts_tablet_manager.cc:1397] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:02:21.285090 5008 raft_consensus.cc:348] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } }
I20260808 05:02:21.285257 5008 raft_consensus.cc:374] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:21.285344 5008 raft_consensus.cc:729] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baf65160af524598aa8729c3918644b4, State: Initialized, Role: FOLLOWER
I20260808 05:02:21.285364 5004 raft_consensus.cc:348] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } }
I20260808 05:02:21.285511 5004 raft_consensus.cc:374] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:21.285472 5008 consensus_queue.cc:260] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } }
I20260808 05:02:21.285548 5004 raft_consensus.cc:729] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f8efd00202c4b2181f1acd4cbabd8c8, State: Initialized, Role: FOLLOWER
I20260808 05:02:21.285655 5004 consensus_queue.cc:260] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } }
I20260808 05:02:21.285727 5008 ts_tablet_manager.cc:1428] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 05:02:21.285774 4729 heartbeater.cc:503] Master 127.3.174.190:36003 was elected leader, sending a full tablet report...
I20260808 05:02:21.285898 5004 ts_tablet_manager.cc:1428] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 05:02:21.285983 4500 heartbeater.cc:503] Master 127.3.174.190:36003 was elected leader, sending a full tablet report...
W20260808 05:02:21.351776 4732 tablet.cc:2369] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:02:21.383200 5015 raft_consensus.cc:482] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:21.383332 5015 raft_consensus.cc:504] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } }
I20260808 05:02:21.383658 5015 leader_election.cc:288] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baf65160af524598aa8729c3918644b4 (127.3.174.130:37485), 5367bd8d26d6475fb86a67591cd7e759 (127.3.174.131:37651)
I20260808 05:02:21.387404 4502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.383831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:37485 (local address 127.3.174.129:48965)
0808 05:02:21.383887 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:21.383900 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:21.383957 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:21.384113 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:21.384114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:21.384116 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:21.384189 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.384190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.385109 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.385111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.386394 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.386398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.387115 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.387116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.387214 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.387218 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:21.387272 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 05:02:21.387316 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":29}
I20260808 05:02:21.387415 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.383961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:48965 (local address 127.3.174.130:37485)
0808 05:02:21.384004 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:21.384006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:21.384015 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:21.384043 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:21.384044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:21.384055 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:21.384101 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:21.384251 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.385000 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.386475 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.387020 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.387095 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.387102 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:21.387301 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 05:02:21.387344 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 05:02:21.387418 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.383904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:40961 (local address 127.3.174.131:37651)
0808 05:02:21.383953 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:21.383956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:21.384134 (+ 178us) server_negotiation.cc:407] Connection header received
0808 05:02:21.384169 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:21.384170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:21.384177 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:21.384216 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:21.384350 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.385042 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.386475 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.387020 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.387094 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.387102 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:21.387322 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 05:02:21.387359 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 05:02:21.387410 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.383832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:37651 (local address 127.3.174.129:40961)
0808 05:02:21.384064 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:21.384073 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:21.384127 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:21.384229 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:21.384230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:21.384232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:21.384299 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.384301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.385120 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.385120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.386408 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:21.386409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.387115 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.387116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:21.387214 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.387219 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 05:02:21.387314 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 05:02:21.387343 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":95,"thread_start_us":107,"threads_started":1}
I20260808 05:02:21.387785 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad84ddb2b774fa093eb2af499a6917c" candidate_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baf65160af524598aa8729c3918644b4" is_pre_election: true
I20260808 05:02:21.387941 4653 raft_consensus.cc:2454] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f8efd00202c4b2181f1acd4cbabd8c8 in term 0.
I20260808 05:02:21.388188 4289 leader_election.cc:302] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f8efd00202c4b2181f1acd4cbabd8c8, baf65160af524598aa8729c3918644b4; no voters:
I20260808 05:02:21.388310 5015 raft_consensus.cc:2781] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:02:21.388368 5015 raft_consensus.cc:482] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:02:21.388319 4885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad84ddb2b774fa093eb2af499a6917c" candidate_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5367bd8d26d6475fb86a67591cd7e759" is_pre_election: true
I20260808 05:02:21.388386 5015 raft_consensus.cc:3037] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:21.388466 4885 raft_consensus.cc:2454] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f8efd00202c4b2181f1acd4cbabd8c8 in term 0.
I20260808 05:02:21.389119 5015 raft_consensus.cc:504] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } }
I20260808 05:02:21.389240 5015 leader_election.cc:288] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [CANDIDATE]: Term 1 election: Requested vote from peers baf65160af524598aa8729c3918644b4 (127.3.174.130:37485), 5367bd8d26d6475fb86a67591cd7e759 (127.3.174.131:37651)
I20260808 05:02:21.389389 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad84ddb2b774fa093eb2af499a6917c" candidate_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baf65160af524598aa8729c3918644b4"
I20260808 05:02:21.389443 4885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad84ddb2b774fa093eb2af499a6917c" candidate_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5367bd8d26d6475fb86a67591cd7e759"
I20260808 05:02:21.389475 4653 raft_consensus.cc:3037] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:21.389509 4885 raft_consensus.cc:3037] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:21.390410 4885 raft_consensus.cc:2454] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f8efd00202c4b2181f1acd4cbabd8c8 in term 1.
I20260808 05:02:21.390410 4653 raft_consensus.cc:2454] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f8efd00202c4b2181f1acd4cbabd8c8 in term 1.
I20260808 05:02:21.390563 4289 leader_election.cc:302] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f8efd00202c4b2181f1acd4cbabd8c8, baf65160af524598aa8729c3918644b4; no voters:
I20260808 05:02:21.390635 5015 raft_consensus.cc:2781] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:21.390759 5015 raft_consensus.cc:686] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 1 LEADER]: Becoming Leader. State: Replica: 2f8efd00202c4b2181f1acd4cbabd8c8, State: Running, Role: LEADER
I20260808 05:02:21.390851 5015 consensus_queue.cc:237] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } }
I20260808 05:02:21.391672 4160 catalog_manager.cc:5495] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f8efd00202c4b2181f1acd4cbabd8c8 (127.3.174.129). New cstate: current_term: 1 leader_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f8efd00202c4b2181f1acd4cbabd8c8" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 39597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 } health_report { overall_health: UNKNOWN } } }
I20260808 05:02:21.415272 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.412910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52496 (local address 127.3.174.129:39597)
0808 05:02:21.412967 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:21.412969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:21.412977 (+ 8us) server_negotiation.cc:407] Connection header received
0808 05:02:21.413011 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:21.413012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:21.413051 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:21.413109 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:21.413228 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.413736 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.414512 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:21.414727 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:21.414791 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:21.414933 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:21.415028 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:21.415108 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:21.415164 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 05:02:21.415174 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 05:02:21.416901 4885 raft_consensus.cc:1261] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [term 1 FOLLOWER]: Refusing update from remote peer 2f8efd00202c4b2181f1acd4cbabd8c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:21.416929 4653 raft_consensus.cc:1261] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [term 1 FOLLOWER]: Refusing update from remote peer 2f8efd00202c4b2181f1acd4cbabd8c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:21.417239 5015 consensus_queue.cc:1035] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baf65160af524598aa8729c3918644b4" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 37485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:21.417410 5018 consensus_queue.cc:1035] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5367bd8d26d6475fb86a67591cd7e759" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 37651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:21.420230 5023 mvcc.cc:205] Tried to move back new op lower bound from 7316133238441304064 to 7316133238337572864. Current Snapshot: MvccSnapshot[applied={T|T < 7316133238441304064}]
I20260808 05:02:21.422187 4160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:21.422268 4160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:21.422940 4160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c61f19196734ca5b49b1917c9900b1c: Sending DeleteTablet for 3 replicas of tablet 4ad84ddb2b774fa093eb2af499a6917c
I20260808 05:02:21.423245 4620 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ad84ddb2b774fa093eb2af499a6917c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:21 UTC) from {username='slave'} at 127.0.0.1:39596
I20260808 05:02:21.423259 4848 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ad84ddb2b774fa093eb2af499a6917c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:21 UTC) from {username='slave'} at 127.0.0.1:59136
I20260808 05:02:21.423475 4379 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ad84ddb2b774fa093eb2af499a6917c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:21 UTC) from {username='slave'} at 127.0.0.1:52492
I20260808 05:02:21.423725 5046 tablet_replica.cc:330] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8: stopping tablet replica
I20260808 05:02:21.423743 5044 tablet_replica.cc:330] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: stopping tablet replica
I20260808 05:02:21.423866 5044 raft_consensus.cc:2229] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:02:21.423887 5046 raft_consensus.cc:2229] T 4ad84ddb2b774fa093eb2af499a6917c P 2f8efd00202c4b2181f1acd4cbabd8c8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:02:21.423975 5044 raft_consensus.cc:2258] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:21.423949 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 4266
I20260808 05:02:21.424389 5044 ts_tablet_manager.cc:1905] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:21.425758 5040 tablet_replica.cc:330] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: stopping tablet replica
I20260808 05:02:21.425872 5040 raft_consensus.cc:2229] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:02:21.425947 5040 raft_consensus.cc:2258] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:21.426069 5044 ts_tablet_manager.cc:1918] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:21.426116 5044 log.cc:1196] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/4ad84ddb2b774fa093eb2af499a6917c
I20260808 05:02:21.426330 5044 ts_tablet_manager.cc:1939] T 4ad84ddb2b774fa093eb2af499a6917c P 5367bd8d26d6475fb86a67591cd7e759: Deleting consensus metadata
I20260808 05:02:21.429474 4139 catalog_manager.cc:4841] TS 5367bd8d26d6475fb86a67591cd7e759 (127.3.174.131:37651): tablet 4ad84ddb2b774fa093eb2af499a6917c (table test-table [id=1dd9731122d9494a894d2fc70eab76b4]) successfully deleted
I20260808 05:02:21.429836 5040 ts_tablet_manager.cc:1905] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:21.431536 5040 ts_tablet_manager.cc:1918] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:21.431618 5040 log.cc:1196] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/4ad84ddb2b774fa093eb2af499a6917c
I20260808 05:02:21.431950 5040 ts_tablet_manager.cc:1939] T 4ad84ddb2b774fa093eb2af499a6917c P baf65160af524598aa8729c3918644b4: Deleting consensus metadata
I20260808 05:02:21.432343 4138 catalog_manager.cc:4841] TS baf65160af524598aa8729c3918644b4 (127.3.174.130:37485): tablet 4ad84ddb2b774fa093eb2af499a6917c (table test-table [id=1dd9731122d9494a894d2fc70eab76b4]) successfully deleted
W20260808 05:02:21.436358 4141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:39597 (error 108)
I20260808 05:02:21.436530 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 4503
I20260808 05:02:21.444278 4993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:21.436674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:39597 (local address 127.0.0.1:52504)
0808 05:02:21.444159 (+ 7485us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:21.444231 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:39597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7456}
W20260808 05:02:21.446848 4141 catalog_manager.cc:4584] TS 2f8efd00202c4b2181f1acd4cbabd8c8 (127.3.174.129:39597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ad84ddb2b774fa093eb2af499a6917c: Network error: Client connection negotiation failed: client connection to 127.3.174.129:39597: connect: Connection refused (error 111)
I20260808 05:02:21.452013 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 4736
I20260808 05:02:21.460954 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 4107
2026-08-08T05:02:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (811 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:21 dist-test-slave-gjw7 krb5kdc[5052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:21 dist-test-slave-gjw7 krb5kdc[5052](info): set up 2 sockets
Aug 08 05:02:21 dist-test-slave-gjw7 krb5kdc[5052](info): commencing operation
krb5kdc: starting...
W20260808 05:02:22.662834 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:22 dist-test-slave-gjw7 krb5kdc[5052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:22.777804 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35935
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:35935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:22.859984 5072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:22.860181 5072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:22.860201 5072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:22.861284 5072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:22.861352 5072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:22.861371 5072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:22.861383 5072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:22.861397 5072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:22.862834 5072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:35935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:22.863023 5072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:22.863197 5072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:22.867801 5080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:22.867918 5079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:22.868010 5072 server_base.cc:1034] running on GCE node
W20260808 05:02:22.867835 5082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:22.868340 5072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:22.868613 5072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:22.869750 5072 hybrid_clock.cc:648] HybridClock initialized: now 1786165342869732 us; error 28 us; skew 500 ppm
Aug 08 05:02:22 dist-test-slave-gjw7 krb5kdc[5052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165342, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:22.872027 5072 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:22.872563 5072 webserver.cc:460] Webserver started at http://127.3.174.190:39651/ using document root <none> and password file <none>
I20260808 05:02:22.872816 5072 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:22.872880 5072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:22.873014 5072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:22.874100 5072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "63602c4de272427eb4206943fde35e35"
format_stamp: "Formatted at 2026-08-08 05:02:22 on dist-test-slave-gjw7"
server_key: "cffb9759e6ed88336f4200ef4071403c"
server_key_iv: "8b3b23a19a694c79be90bf6d4c5fd20e"
server_key_version: "encryptionkey@0"
I20260808 05:02:22.874497 5072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "63602c4de272427eb4206943fde35e35"
format_stamp: "Formatted at 2026-08-08 05:02:22 on dist-test-slave-gjw7"
server_key: "cffb9759e6ed88336f4200ef4071403c"
server_key_iv: "8b3b23a19a694c79be90bf6d4c5fd20e"
server_key_version: "encryptionkey@0"
I20260808 05:02:22.875742 5072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 05:02:22.876384 5093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:22.876554 5072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:22.876639 5072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "63602c4de272427eb4206943fde35e35"
format_stamp: "Formatted at 2026-08-08 05:02:22 on dist-test-slave-gjw7"
server_key: "cffb9759e6ed88336f4200ef4071403c"
server_key_iv: "8b3b23a19a694c79be90bf6d4c5fd20e"
server_key_version: "encryptionkey@0"
I20260808 05:02:22.876703 5072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:22.889760 5072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:22.891477 5072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:22.891613 5072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:22.895618 5072 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:35935
I20260808 05:02:22.895670 5181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:35935 every 8 connection(s)
I20260808 05:02:22.896006 5072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:22.896872 5185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:22.899223 5185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35: Bootstrap starting.
I20260808 05:02:22.899785 5185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:22.900071 5185 log.cc:824] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:22.900701 5185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35: No bootstrap required, opened a new log
I20260808 05:02:22.902081 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 5072
I20260808 05:02:22.902185 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:22.902374 3770 external_mini_cluster.cc:1428] Setting key e5d1bd73ccc7a21945682ac56a5b6a16
I20260808 05:02:22.902365 5185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63602c4de272427eb4206943fde35e35" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35935 } }
I20260808 05:02:22.902524 5185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:22.902561 5185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63602c4de272427eb4206943fde35e35, State: Initialized, Role: FOLLOWER
I20260808 05:02:22.902714 5185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63602c4de272427eb4206943fde35e35" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35935 } }
I20260808 05:02:22.902806 5185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:22.902848 5185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:22.902891 5185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:22.903676 5185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63602c4de272427eb4206943fde35e35" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35935 } }
I20260808 05:02:22.903780 5185 leader_election.cc:302] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63602c4de272427eb4206943fde35e35; no voters:
I20260808 05:02:22.903954 5185 leader_election.cc:288] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:22.903997 5192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:22.904143 5192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [term 1 LEADER]: Becoming Leader. State: Replica: 63602c4de272427eb4206943fde35e35, State: Running, Role: LEADER
I20260808 05:02:22.904204 5185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:22.904234 5192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63602c4de272427eb4206943fde35e35" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35935 } }
I20260808 05:02:22.904574 5192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63602c4de272427eb4206943fde35e35. Latest consensus state: current_term: 1 leader_uuid: "63602c4de272427eb4206943fde35e35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63602c4de272427eb4206943fde35e35" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35935 } } }
I20260808 05:02:22.904649 5192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:22.904803 5196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63602c4de272427eb4206943fde35e35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63602c4de272427eb4206943fde35e35" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35935 } } }
I20260808 05:02:22.904870 5196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:22.904970 5201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:22.905449 5201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:22.907011 5201 catalog_manager.cc:1347] Generated new cluster ID: 8decf15785dc45098269bc4de1dcad1f
I20260808 05:02:22.907061 5201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 05:02:22 dist-test-slave-gjw7 krb5kdc[5052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:22.913266 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:22.902827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.3.174.190:35935)
0808 05:02:22.903105 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:22.903114 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:22.903139 (+ 25us) server_negotiation.cc:407] Connection header received
0808 05:02:22.903248 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:22.903254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:22.903307 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:22.903419 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:22.904115 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:22.905527 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:22.906515 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:22.907082 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:22.907195 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:22.909906 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:22.909910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:22.909911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:22.909923 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:22.911723 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:22.912127 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:22.912130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:22.912131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:22.912236 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:22.912435 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:22.912435 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:22.912436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:22.912706 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:22.912860 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:22.912947 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 05:02:22.913074 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":231,"thread_start_us":86,"threads_started":1}
I20260808 05:02:22.927354 5201 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:22.927872 5201 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:22.934463 5201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35: Generated new TSK 0
I20260808 05:02:22.934639 5201 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.174.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
Entry for principal kudu/127.3.174.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab.
I20260808 05:02:22.965586 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35935
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:23.048841 5220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:23.049048 5220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:23.049070 5220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:23.050221 5220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:23.050272 5220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:23.050299 5220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:02:23.051734 5220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.129.keytab
--principal=kudu/127.3.174.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:23.051976 5220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:23.052193 5220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:23.057000 5229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:23.057006 5232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:23.057021 5227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:23.057253 5220 server_base.cc:1034] running on GCE node
I20260808 05:02:23.057472 5220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:23.057686 5220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:23.058821 5220 hybrid_clock.cc:648] HybridClock initialized: now 1786165343058807 us; error 28 us; skew 500 ppm
Aug 08 05:02:23 dist-test-slave-gjw7 krb5kdc[5052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165343, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:23.061281 5220 init.cc:377] Logged in from keytab as kudu/127.3.174.129@KRBTEST.COM (short username kudu)
I20260808 05:02:23.061725 5220 webserver.cc:460] Webserver started at http://127.3.174.129:43475/ using document root <none> and password file <none>
I20260808 05:02:23.061959 5220 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:23.062002 5220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:23.062129 5220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:23.063141 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "19bdce72856a465681646d7c6623bbc0"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "5239cbe29bce72e846186fe3160019b4"
server_key_iv: "b366c6eaf276ab9c21e44b9a1ff87320"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.063490 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "19bdce72856a465681646d7c6623bbc0"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "5239cbe29bce72e846186fe3160019b4"
server_key_iv: "b366c6eaf276ab9c21e44b9a1ff87320"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.064823 5220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:23.065636 5243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.065766 5220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.065851 5220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "19bdce72856a465681646d7c6623bbc0"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "5239cbe29bce72e846186fe3160019b4"
server_key_iv: "b366c6eaf276ab9c21e44b9a1ff87320"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.065918 5220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:23.081858 5220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:23.083571 5220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:23.083710 5220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:23.083922 5220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:23.084234 5220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:23.084270 5220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.084293 5220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:23.084316 5220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.089892 5220 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:44579
I20260808 05:02:23.089954 5449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:44579 every 8 connection(s)
I20260808 05:02:23.090260 5220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:02:23.090741 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 5220
I20260808 05:02:23.090858 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:02:23.091114 3770 external_mini_cluster.cc:1428] Setting key 7813e1c8b1e458c26c3245c93c2a339e
Aug 08 05:02:23 dist-test-slave-gjw7 krb5kdc[5052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165343, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.129@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.174.130@KRBTEST.COM; defaulting to no policy
I20260808 05:02:23.100109 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.090993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:58093 (local address 127.3.174.190:35935)
0808 05:02:23.091063 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.091066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.091279 (+ 213us) server_negotiation.cc:407] Connection header received
0808 05:02:23.091886 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.091887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.091901 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.091954 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:23.093180 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.093776 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.094773 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.094970 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.095028 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.097412 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:23.097414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:23.097414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:23.097420 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:23.098766 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:23.099149 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:23.099150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:23.099151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:23.099205 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:23.099459 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:23.099460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:23.099460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:23.099568 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:23.099625 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.099917 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.100012 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 05:02:23.100232 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.090927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35935 (local address 127.3.174.129:58093)
0808 05:02:23.091099 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:23.091189 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:23.091793 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:23.092037 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:23.092042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:23.092806 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:23.093071 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.093078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.093876 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.093877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.094674 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.094677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.095042 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.095043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.095180 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.095689 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:23.095699 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:23.097286 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:23.098852 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:23.098860 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:23.098862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:23.099080 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:23.099287 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:23.099288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:23.099288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:23.099397 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:23.099624 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:23.099631 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:23.099769 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 05:02:23.099857 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 05:02:23.100017 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260808 05:02:23.100716 5450 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35935
I20260808 05:02:23.100816 5450 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:23.100991 5450 heartbeater.cc:511] Master 127.3.174.190:35935 requested a full tablet report, sending...
I20260808 05:02:23.101568 5120 ts_manager.cc:194] Registered new tserver with Master: 19bdce72856a465681646d7c6623bbc0 (127.3.174.129:44579)
I20260808 05:02:23.102247 5120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.129@KRBTEST.COM'} at 127.3.174.129:58093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
Entry for principal kudu/127.3.174.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab.
I20260808 05:02:23.115913 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35935
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:23.200037 5455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:23.200238 5455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:23.200261 5455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:23.201444 5455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:23.201483 5455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:23.201507 5455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:02:23.202930 5455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.130.keytab
--principal=kudu/127.3.174.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:23.203158 5455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:23.203392 5455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 05:02:23.208427 5455 server_base.cc:1034] running on GCE node
W20260808 05:02:23.208415 5464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:23.208395 5466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:23.208375 5463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:23.208809 5455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:23.209026 5455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:23.210206 5455 hybrid_clock.cc:648] HybridClock initialized: now 1786165343210154 us; error 30 us; skew 500 ppm
Aug 08 05:02:23 dist-test-slave-gjw7 krb5kdc[5052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165343, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:23.212598 5455 init.cc:377] Logged in from keytab as kudu/127.3.174.130@KRBTEST.COM (short username kudu)
I20260808 05:02:23.213009 5455 webserver.cc:460] Webserver started at http://127.3.174.130:35355/ using document root <none> and password file <none>
I20260808 05:02:23.213217 5455 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:23.213258 5455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:23.213452 5455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:23.214501 5455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "838ddd8c6d54457995a418bfd5add413"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "2877f61dcf8fbdec9ce12d4e94a4c4b3"
server_key_iv: "7bb7de8b59a2fcc7999d9b44ecb1ef22"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.214792 5455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "838ddd8c6d54457995a418bfd5add413"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "2877f61dcf8fbdec9ce12d4e94a4c4b3"
server_key_iv: "7bb7de8b59a2fcc7999d9b44ecb1ef22"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.215971 5455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 05:02:23.216646 5476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.216804 5455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:23.216887 5455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "838ddd8c6d54457995a418bfd5add413"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "2877f61dcf8fbdec9ce12d4e94a4c4b3"
server_key_iv: "7bb7de8b59a2fcc7999d9b44ecb1ef22"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.216943 5455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:23.236343 5455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:23.238097 5455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:23.238209 5455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:23.238406 5455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:23.238713 5455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:23.238747 5455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.238790 5455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:23.238812 5455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.244222 5455 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:38249
I20260808 05:02:23.244283 5666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:38249 every 8 connection(s)
I20260808 05:02:23.244572 5455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
Aug 08 05:02:23 dist-test-slave-gjw7 krb5kdc[5052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165343, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.130@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:23.251183 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 5455
I20260808 05:02:23.251309 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:02:23.251538 3770 external_mini_cluster.cc:1428] Setting key 025ddc37e5a597c6b6cb0764be8eee99
I20260808 05:02:23.253384 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.245275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:46885 (local address 127.3.174.190:35935)
0808 05:02:23.245338 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.245340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.245555 (+ 215us) server_negotiation.cc:407] Connection header received
0808 05:02:23.246041 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.246042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.246052 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.246101 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:23.246973 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.247472 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.248389 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.248584 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.248632 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.250573 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:23.250575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:23.250575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:23.250582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:23.252031 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:23.252443 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:23.252445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:23.252445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:23.252566 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:23.252774 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:23.252775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:23.252775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:23.252901 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:23.252966 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.253179 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.253263 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 05:02:23.253701 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.245249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35935 (local address 127.3.174.130:46885)
0808 05:02:23.245440 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:23.245477 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:23.245964 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:23.246155 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:23.246159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:23.246701 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:23.246904 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.246910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.247535 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.247536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.248323 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.248326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.248639 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.248639 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.248737 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.249165 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:23.249173 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:23.250489 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:23.252109 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:23.252117 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:23.252118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:23.252375 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:23.252629 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:23.252630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:23.252630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:23.252721 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:23.252958 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:23.252961 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:23.253071 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 05:02:23.253150 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 05:02:23.253268 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260808 05:02:23.254169 5667 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35935
I20260808 05:02:23.254267 5667 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:23.254446 5667 heartbeater.cc:511] Master 127.3.174.190:35935 requested a full tablet report, sending...
I20260808 05:02:23.254817 5120 ts_manager.cc:194] Registered new tserver with Master: 838ddd8c6d54457995a418bfd5add413 (127.3.174.130:38249)
I20260808 05:02:23.255345 5120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.130@KRBTEST.COM'} at 127.3.174.130:46885
WARNING: no policy specified for kudu/127.3.174.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
Entry for principal kudu/127.3.174.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab.
I20260808 05:02:23.276572 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35935
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:23.360373 5675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:23.360560 5675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:23.360581 5675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:23.361729 5675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:23.361769 5675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:23.361794 5675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:02:23.363238 5675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.131.keytab
--principal=kudu/127.3.174.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:35935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:23.363430 5675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:23.363615 5675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:23.368460 5686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:23.368454 5683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:23.368456 5680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:23.368592 5675 server_base.cc:1034] running on GCE node
I20260808 05:02:23.368857 5675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:23.369098 5675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:23.370240 5675 hybrid_clock.cc:648] HybridClock initialized: now 1786165343370228 us; error 29 us; skew 500 ppm
Aug 08 05:02:23 dist-test-slave-gjw7 krb5kdc[5052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165343, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:23.372633 5675 init.cc:377] Logged in from keytab as kudu/127.3.174.131@KRBTEST.COM (short username kudu)
I20260808 05:02:23.373049 5675 webserver.cc:460] Webserver started at http://127.3.174.131:39253/ using document root <none> and password file <none>
I20260808 05:02:23.373248 5675 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:23.373310 5675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:23.373433 5675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:23.374433 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "47bca05cb72a4d0dbdf173d1cde4c789"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "673fd46aac0feef494b6254cd3805d21"
server_key_iv: "c85fe0b3061f3b61f86f1e5e31dda861"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.374730 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "47bca05cb72a4d0dbdf173d1cde4c789"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "673fd46aac0feef494b6254cd3805d21"
server_key_iv: "c85fe0b3061f3b61f86f1e5e31dda861"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.375916 5675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 05:02:23.376569 5698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.376711 5675 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:23.376806 5675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "47bca05cb72a4d0dbdf173d1cde4c789"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "673fd46aac0feef494b6254cd3805d21"
server_key_iv: "c85fe0b3061f3b61f86f1e5e31dda861"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.376863 5675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:23.397028 5675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:23.399180 5675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:23.399293 5675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:23.399497 5675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:23.399760 5675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:23.399797 5675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.399823 5675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:23.399837 5675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.405575 5675 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:42961
I20260808 05:02:23.405642 5888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:42961 every 8 connection(s)
I20260808 05:02:23.406092 5675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:02:23.411720 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 5675
I20260808 05:02:23.411818 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
Aug 08 05:02:23 dist-test-slave-gjw7 krb5kdc[5052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165343, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.131@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:23.411999 3770 external_mini_cluster.cc:1428] Setting key 4d15fe408625c4debe9c0f66f9aa770b
I20260808 05:02:23.414945 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.406769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:35313 (local address 127.3.174.190:35935)
0808 05:02:23.406829 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.406831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.407021 (+ 190us) server_negotiation.cc:407] Connection header received
0808 05:02:23.407957 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.407958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.407969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.408018 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:23.408597 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.409085 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.410083 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.410304 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.410364 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.412436 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:23.412438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:23.412438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:23.412446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:23.413842 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:23.414188 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:23.414188 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:23.414189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:23.414233 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:23.414458 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:23.414459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:23.414459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:23.414565 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:23.414614 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.414820 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.414878 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 05:02:23.415123 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.406755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:35935 (local address 127.3.174.131:35313)
0808 05:02:23.406915 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:23.406953 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:23.407868 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:23.408074 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:23.408078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:23.408331 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:23.408524 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.408528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.409146 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.409146 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.409987 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.409990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.410370 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.410371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.410465 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.410900 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:23.410908 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:23.412372 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:23.413909 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:23.413917 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:23.413919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:23.414131 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:23.414275 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:23.414276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:23.414276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:23.414410 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:23.414612 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:23.414616 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:23.414724 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 05:02:23.414799 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 05:02:23.414919 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260808 05:02:23.415542 5889 heartbeater.cc:347] Connected to a master server at 127.3.174.190:35935
I20260808 05:02:23.415627 5889 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:23.415781 5889 heartbeater.cc:511] Master 127.3.174.190:35935 requested a full tablet report, sending...
I20260808 05:02:23.416134 5120 ts_manager.cc:194] Registered new tserver with Master: 47bca05cb72a4d0dbdf173d1cde4c789 (127.3.174.131:42961)
I20260808 05:02:23.416477 5120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.174.131@KRBTEST.COM'} at 127.3.174.131:35313
I20260808 05:02:23.423652 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:02:23.430454 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.424985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40280 (local address 127.3.174.190:35935)
0808 05:02:23.425039 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.425041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.425145 (+ 104us) server_negotiation.cc:407] Connection header received
0808 05:02:23.425208 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.425209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.425218 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.425262 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:23.425801 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.426269 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.426964 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.427147 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.427199 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.428326 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:23.428327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:23.428328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:23.428335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:23.429442 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:23.429738 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:23.429740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:23.429740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:23.429793 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:23.429952 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:23.429953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:23.429953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:23.430095 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:23.430184 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.430286 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.430367 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 05:02:23.431778 5120 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:02:23.433333 5120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:02:23.439339 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.435346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:42961 (local address 127.0.0.1:39322)
0808 05:02:23.435454 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:23.435492 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:23.435555 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:23.435865 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:23.435868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:23.435870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:23.435946 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.435948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.437197 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.437198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.438245 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.438248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.439035 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.439036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.439123 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.439138 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 05:02:23.439210 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 05:02:23.439264 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":42,"thread_start_us":59,"threads_started":1}
I20260808 05:02:23.439363 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.435519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39322 (local address 127.3.174.131:42961)
0808 05:02:23.435697 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.435703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.435715 (+ 12us) server_negotiation.cc:407] Connection header received
0808 05:02:23.435754 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.435756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.435771 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.435845 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:23.436027 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.437077 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.438320 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.438952 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.439027 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.439085 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.439233 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.439286 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260808 05:02:23.439620 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.435363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:44579 (local address 127.0.0.1:33092)
0808 05:02:23.435688 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:23.435698 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:23.435749 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:23.436338 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:23.436339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:23.436341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:23.436384 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.436386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.437591 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.437592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.438625 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.438627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.439414 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.439414 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.439473 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.439477 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:23.439520 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 05:02:23.439558 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":1,"thread_start_us":45,"threads_started":1}
I20260808 05:02:23.439853 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.435831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38390 (local address 127.3.174.130:38249)
0808 05:02:23.436011 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.436016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.436025 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:23.436062 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.436064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.436080 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.436152 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:23.436377 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.437079 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.438514 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.439192 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.439254 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.439307 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.439658 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.439776 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260808 05:02:23.439875 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.435624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.3.174.129:44579)
0808 05:02:23.436105 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.436109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.436119 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:23.436157 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.436160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.436174 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.436246 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:23.436461 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.437493 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.438693 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.439249 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.439319 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.439376 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.439766 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.439813 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":95,"threads_started":1}
I20260808 05:02:23.440238 5790 tablet_service.cc:1467] Processing CreateTablet for tablet 6fe9bc49a9b0447891915d2a97259630 (DEFAULT_TABLE table=test-table [id=a6175eebbbf74f27b386d7a9b8feb5d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:23.440500 5327 tablet_service.cc:1467] Processing CreateTablet for tablet 6fe9bc49a9b0447891915d2a97259630 (DEFAULT_TABLE table=test-table [id=a6175eebbbf74f27b386d7a9b8feb5d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:23.440610 5790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fe9bc49a9b0447891915d2a97259630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:23.440820 5327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fe9bc49a9b0447891915d2a97259630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:23.441398 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.435350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:38249 (local address 127.0.0.1:38390)
0808 05:02:23.435640 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:23.435648 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:23.435682 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:23.436229 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:23.436230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:23.436232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:23.436311 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.436313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.437168 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.437169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.438433 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.438436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.439343 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.439344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.439413 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.439586 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 05:02:23.439639 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 05:02:23.439681 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":35,"thread_start_us":65,"threads_started":1}
I20260808 05:02:23.442006 5559 tablet_service.cc:1467] Processing CreateTablet for tablet 6fe9bc49a9b0447891915d2a97259630 (DEFAULT_TABLE table=test-table [id=a6175eebbbf74f27b386d7a9b8feb5d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:23.442296 5559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fe9bc49a9b0447891915d2a97259630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:23.443977 5908 tablet_bootstrap.cc:492] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0: Bootstrap starting.
I20260808 05:02:23.444252 5909 tablet_bootstrap.cc:492] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: Bootstrap starting.
I20260808 05:02:23.444725 5908 tablet_bootstrap.cc:654] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:23.444794 5909 tablet_bootstrap.cc:654] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:23.445137 5910 tablet_bootstrap.cc:492] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: Bootstrap starting.
I20260808 05:02:23.445334 5908 log.cc:824] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:23.445369 5909 log.cc:824] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:23.445920 5910 tablet_bootstrap.cc:654] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:23.446077 5908 tablet_bootstrap.cc:492] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0: No bootstrap required, opened a new log
I20260808 05:02:23.446087 5909 tablet_bootstrap.cc:492] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: No bootstrap required, opened a new log
I20260808 05:02:23.446153 5908 ts_tablet_manager.cc:1397] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:02:23.446153 5909 ts_tablet_manager.cc:1397] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 05:02:23.446403 5910 log.cc:824] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:23.447001 5910 tablet_bootstrap.cc:492] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: No bootstrap required, opened a new log
I20260808 05:02:23.447060 5910 ts_tablet_manager.cc:1397] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:02:23.448037 5909 raft_consensus.cc:348] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } }
I20260808 05:02:23.448037 5908 raft_consensus.cc:348] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } }
I20260808 05:02:23.448222 5908 raft_consensus.cc:374] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:23.448222 5909 raft_consensus.cc:374] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:23.448259 5909 raft_consensus.cc:729] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bca05cb72a4d0dbdf173d1cde4c789, State: Initialized, Role: FOLLOWER
I20260808 05:02:23.448259 5908 raft_consensus.cc:729] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19bdce72856a465681646d7c6623bbc0, State: Initialized, Role: FOLLOWER
I20260808 05:02:23.448378 5909 consensus_queue.cc:260] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } }
I20260808 05:02:23.448395 5908 consensus_queue.cc:260] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } }
I20260808 05:02:23.448447 5910 raft_consensus.cc:348] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } }
I20260808 05:02:23.448628 5910 raft_consensus.cc:374] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:23.448665 5908 ts_tablet_manager.cc:1428] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 05:02:23.448665 5909 ts_tablet_manager.cc:1428] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 05:02:23.448673 5910 raft_consensus.cc:729] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838ddd8c6d54457995a418bfd5add413, State: Initialized, Role: FOLLOWER
I20260808 05:02:23.448786 5889 heartbeater.cc:503] Master 127.3.174.190:35935 was elected leader, sending a full tablet report...
I20260808 05:02:23.448804 5450 heartbeater.cc:503] Master 127.3.174.190:35935 was elected leader, sending a full tablet report...
I20260808 05:02:23.448805 5910 consensus_queue.cc:260] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } }
I20260808 05:02:23.449075 5910 ts_tablet_manager.cc:1428] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:02:23.449182 5667 heartbeater.cc:503] Master 127.3.174.190:35935 was elected leader, sending a full tablet report...
W20260808 05:02:23.495136 5669 tablet.cc:2369] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 05:02:23.590992 5451 tablet.cc:2369] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 05:02:23.656689 5890 tablet.cc:2369] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:02:23.697594 5922 raft_consensus.cc:482] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:23.697732 5922 raft_consensus.cc:504] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } }
I20260808 05:02:23.698148 5922 leader_election.cc:288] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47bca05cb72a4d0dbdf173d1cde4c789 (127.3.174.131:42961), 19bdce72856a465681646d7c6623bbc0 (127.3.174.129:44579)
I20260808 05:02:23.702040 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.698458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:37707 (local address 127.3.174.131:42961)
0808 05:02:23.698509 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.698511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.698523 (+ 12us) server_negotiation.cc:407] Connection header received
0808 05:02:23.698558 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.698559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.698569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.698627 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:23.698806 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.699546 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.701049 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.701649 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.701725 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.701734 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.701926 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.701965 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 05:02:23.702040 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.698444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:39573 (local address 127.3.174.129:44579)
0808 05:02:23.698515 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.698517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.698672 (+ 155us) server_negotiation.cc:407] Connection header received
0808 05:02:23.698704 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.698705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.698718 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.698760 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:23.698898 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.699581 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.701049 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.701649 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.701725 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.701734 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.701926 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.701965 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 05:02:23.702075 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.698385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:42961 (local address 127.3.174.130:37707)
0808 05:02:23.698447 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:23.698458 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:23.698502 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:23.698645 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:23.698646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:23.698648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:23.698726 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.698728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.699640 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.699641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.700955 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.700959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.701744 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.701746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.701845 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.701847 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 05:02:23.701921 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 05:02:23.701971 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":76}
I20260808 05:02:23.702083 5926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.698393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:44579 (local address 127.3.174.130:39573)
0808 05:02:23.698589 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:23.698598 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:23.698656 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:23.698782 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:23.698783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:23.698784 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:23.698843 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.698844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.699642 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.699642 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.700955 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:23.700959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.701745 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.701746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:23.701840 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.701846 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 05:02:23.701921 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 05:02:23.701969 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":48,"thread_start_us":100,"threads_started":1}
I20260808 05:02:23.702466 5819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe9bc49a9b0447891915d2a97259630" candidate_uuid: "838ddd8c6d54457995a418bfd5add413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" is_pre_election: true
I20260808 05:02:23.702538 5365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe9bc49a9b0447891915d2a97259630" candidate_uuid: "838ddd8c6d54457995a418bfd5add413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19bdce72856a465681646d7c6623bbc0" is_pre_election: true
I20260808 05:02:23.702605 5819 raft_consensus.cc:2454] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838ddd8c6d54457995a418bfd5add413 in term 0.
I20260808 05:02:23.702732 5365 raft_consensus.cc:2454] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838ddd8c6d54457995a418bfd5add413 in term 0.
I20260808 05:02:23.702844 5479 leader_election.cc:302] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47bca05cb72a4d0dbdf173d1cde4c789, 838ddd8c6d54457995a418bfd5add413; no voters:
I20260808 05:02:23.703004 5922 raft_consensus.cc:2781] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:02:23.703065 5922 raft_consensus.cc:482] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:02:23.703086 5922 raft_consensus.cc:3037] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:23.703822 5922 raft_consensus.cc:504] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } }
I20260808 05:02:23.703951 5922 leader_election.cc:288] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [CANDIDATE]: Term 1 election: Requested vote from peers 47bca05cb72a4d0dbdf173d1cde4c789 (127.3.174.131:42961), 19bdce72856a465681646d7c6623bbc0 (127.3.174.129:44579)
I20260808 05:02:23.704239 5365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe9bc49a9b0447891915d2a97259630" candidate_uuid: "838ddd8c6d54457995a418bfd5add413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19bdce72856a465681646d7c6623bbc0"
I20260808 05:02:23.704275 5819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe9bc49a9b0447891915d2a97259630" candidate_uuid: "838ddd8c6d54457995a418bfd5add413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47bca05cb72a4d0dbdf173d1cde4c789"
I20260808 05:02:23.704324 5365 raft_consensus.cc:3037] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:23.704347 5819 raft_consensus.cc:3037] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:23.705060 5365 raft_consensus.cc:2454] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838ddd8c6d54457995a418bfd5add413 in term 1.
I20260808 05:02:23.705132 5819 raft_consensus.cc:2454] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838ddd8c6d54457995a418bfd5add413 in term 1.
I20260808 05:02:23.705252 5478 leader_election.cc:302] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19bdce72856a465681646d7c6623bbc0, 838ddd8c6d54457995a418bfd5add413; no voters:
I20260808 05:02:23.705395 5922 raft_consensus.cc:2781] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:23.705546 5922 raft_consensus.cc:686] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 1 LEADER]: Becoming Leader. State: Replica: 838ddd8c6d54457995a418bfd5add413, State: Running, Role: LEADER
I20260808 05:02:23.705646 5922 consensus_queue.cc:237] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } }
I20260808 05:02:23.706537 5118 catalog_manager.cc:5495] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838ddd8c6d54457995a418bfd5add413 (127.3.174.130). New cstate: current_term: 1 leader_uuid: "838ddd8c6d54457995a418bfd5add413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838ddd8c6d54457995a418bfd5add413" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 38249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 } health_report { overall_health: UNKNOWN } } }
I20260808 05:02:23.721943 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.719526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.3.174.130:38249)
0808 05:02:23.719584 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.719586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.719595 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:23.719654 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.719655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.719676 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.719735 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:23.719882 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.720407 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.721145 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.721392 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.721444 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.721603 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:23.721716 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:23.721788 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.721828 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.721848 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 05:02:23.723752 5365 raft_consensus.cc:1261] T 6fe9bc49a9b0447891915d2a97259630 P 19bdce72856a465681646d7c6623bbc0 [term 1 FOLLOWER]: Refusing update from remote peer 838ddd8c6d54457995a418bfd5add413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:23.723764 5819 raft_consensus.cc:1261] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [term 1 FOLLOWER]: Refusing update from remote peer 838ddd8c6d54457995a418bfd5add413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:23.724098 5927 consensus_queue.cc:1035] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19bdce72856a465681646d7c6623bbc0" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 44579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:23.724237 5922 consensus_queue.cc:1035] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47bca05cb72a4d0dbdf173d1cde4c789" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 42961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:23.726928 5932 mvcc.cc:205] Tried to move back new op lower bound from 7316133247889825792 to 7316133247818805248. Current Snapshot: MvccSnapshot[applied={T|T < 7316133247889825792}]
I20260808 05:02:23.727041 5936 mvcc.cc:205] Tried to move back new op lower bound from 7316133247889825792 to 7316133247818805248. Current Snapshot: MvccSnapshot[applied={T|T < 7316133247889825792}]
I20260808 05:02:23.729476 5120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:23.729565 5120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:23.730283 5120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63602c4de272427eb4206943fde35e35: Sending DeleteTablet for 3 replicas of tablet 6fe9bc49a9b0447891915d2a97259630
I20260808 05:02:23.730602 5327 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fe9bc49a9b0447891915d2a97259630 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:23 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:33092
I20260808 05:02:23.730859 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 5220
I20260808 05:02:23.730844 5790 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fe9bc49a9b0447891915d2a97259630 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:23 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:39322
I20260808 05:02:23.736893 5950 tablet_replica.cc:330] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: stopping tablet replica
I20260808 05:02:23.738299 5559 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fe9bc49a9b0447891915d2a97259630 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:23 UTC) from {username='kudu', principal='kudu/127.3.174.190@KRBTEST.COM'} at 127.0.0.1:38390
I20260808 05:02:23.738405 5950 raft_consensus.cc:2229] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:02:23.738591 5950 raft_consensus.cc:2258] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:23.739491 5951 tablet_replica.cc:330] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: stopping tablet replica
I20260808 05:02:23.739658 5951 raft_consensus.cc:2229] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:02:23.739986 5951 raft_consensus.cc:2258] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:23.740025 5950 ts_tablet_manager.cc:1905] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:23.740497 5951 ts_tablet_manager.cc:1905] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:23.742060 5950 ts_tablet_manager.cc:1918] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:23.742195 5950 log.cc:1196] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/6fe9bc49a9b0447891915d2a97259630
I20260808 05:02:23.742247 5951 ts_tablet_manager.cc:1918] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:23.742352 5951 log.cc:1196] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/6fe9bc49a9b0447891915d2a97259630
I20260808 05:02:23.742538 5950 ts_tablet_manager.cc:1939] T 6fe9bc49a9b0447891915d2a97259630 P 47bca05cb72a4d0dbdf173d1cde4c789: Deleting consensus metadata
I20260808 05:02:23.742621 5951 ts_tablet_manager.cc:1939] T 6fe9bc49a9b0447891915d2a97259630 P 838ddd8c6d54457995a418bfd5add413: Deleting consensus metadata
I20260808 05:02:23.743055 5099 catalog_manager.cc:4841] TS 47bca05cb72a4d0dbdf173d1cde4c789 (127.3.174.131:42961): tablet 6fe9bc49a9b0447891915d2a97259630 (table test-table [id=a6175eebbbf74f27b386d7a9b8feb5d0]) successfully deleted
I20260808 05:02:23.743216 5100 catalog_manager.cc:4841] TS 838ddd8c6d54457995a418bfd5add413 (127.3.174.130:38249): tablet 6fe9bc49a9b0447891915d2a97259630 (table test-table [id=a6175eebbbf74f27b386d7a9b8feb5d0]) successfully deleted
W20260808 05:02:23.748376 5098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:44579 (error 108)
I20260808 05:02:23.748637 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 5455
I20260808 05:02:23.750895 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.748772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:44579 (local address 127.0.0.1:33100)
0808 05:02:23.750779 (+ 2007us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:23.750849 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:44579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1975}
W20260808 05:02:23.757925 5098 catalog_manager.cc:4584] TS 19bdce72856a465681646d7c6623bbc0 (127.3.174.129:44579): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6fe9bc49a9b0447891915d2a97259630: Network error: Client connection negotiation failed: client connection to 127.3.174.129:44579: connect: Connection refused (error 111)
I20260808 05:02:23.771921 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 5675
I20260808 05:02:23.780634 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 5072
2026-08-08T05:02:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2319 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T05:02:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:23Z Disabled control of system clock
I20260808 05:02:23.815900 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33613
--webserver_interface=127.3.174.190
--webserver_port=0
--builtin_ntp_servers=127.3.174.148:46861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:33613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:23.899341 5956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:23.899576 5956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:23.899607 5956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:23.900823 5956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:23.900866 5956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:23.900884 5956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:23.900899 5956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:23.900915 5956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:23.902348 5956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:46861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:33613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:33613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:23.902601 5956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:23.902884 5956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:23.907582 5966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:23.907591 5969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:23.907816 5956 server_base.cc:1034] running on GCE node
W20260808 05:02:23.907847 5967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:23.908119 5956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:23.908409 5956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:23.909545 5956 hybrid_clock.cc:648] HybridClock initialized: now 1786165343909533 us; error 35 us; skew 500 ppm
I20260808 05:02:23.910672 5956 webserver.cc:460] Webserver started at http://127.3.174.190:34575/ using document root <none> and password file <none>
I20260808 05:02:23.910902 5956 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:23.910952 5956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:23.911082 5956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:23.912130 5956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "58a400e59a204ce692bbcf8da318ae16"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "3765bf9a90dd2a627ebd93d115ea51b2"
server_key_iv: "51116056aa575328ea0dd84fa9eb31e3"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.912472 5956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "58a400e59a204ce692bbcf8da318ae16"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "3765bf9a90dd2a627ebd93d115ea51b2"
server_key_iv: "51116056aa575328ea0dd84fa9eb31e3"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.913712 5956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:23.914338 5983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:23.914486 5956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:23.914570 5956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "58a400e59a204ce692bbcf8da318ae16"
format_stamp: "Formatted at 2026-08-08 05:02:23 on dist-test-slave-gjw7"
server_key: "3765bf9a90dd2a627ebd93d115ea51b2"
server_key_iv: "51116056aa575328ea0dd84fa9eb31e3"
server_key_version: "encryptionkey@0"
I20260808 05:02:23.914630 5956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:23.928220 5956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:23.928547 5956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:23.928681 5956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:23.932582 5956 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:33613
I20260808 05:02:23.932616 6071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:33613 every 8 connection(s)
I20260808 05:02:23.932945 5956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:23.933997 6073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:23.936355 6073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16: Bootstrap starting.
I20260808 05:02:23.936919 6073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:23.937237 6073 log.cc:824] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:23.937898 6073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16: No bootstrap required, opened a new log
I20260808 05:02:23.939512 6073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a400e59a204ce692bbcf8da318ae16" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33613 } }
I20260808 05:02:23.939654 6073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:23.939687 6073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a400e59a204ce692bbcf8da318ae16, State: Initialized, Role: FOLLOWER
I20260808 05:02:23.939843 6073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a400e59a204ce692bbcf8da318ae16" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33613 } }
I20260808 05:02:23.939919 6073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:23.939945 6073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:23.939992 6073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:23.940258 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 5956
I20260808 05:02:23.940362 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:23.940572 3770 external_mini_cluster.cc:1428] Setting key 1d4f95b0baf700485497b9fb3fc07b98
I20260808 05:02:23.940670 6073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a400e59a204ce692bbcf8da318ae16" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33613 } }
I20260808 05:02:23.940784 6073 leader_election.cc:302] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58a400e59a204ce692bbcf8da318ae16; no voters:
I20260808 05:02:23.940940 6073 leader_election.cc:288] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:23.940971 6079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:23.941171 6079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [term 1 LEADER]: Becoming Leader. State: Replica: 58a400e59a204ce692bbcf8da318ae16, State: Running, Role: LEADER
I20260808 05:02:23.941324 6079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a400e59a204ce692bbcf8da318ae16" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33613 } }
I20260808 05:02:23.941318 6073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:23.941846 6082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58a400e59a204ce692bbcf8da318ae16. Latest consensus state: current_term: 1 leader_uuid: "58a400e59a204ce692bbcf8da318ae16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a400e59a204ce692bbcf8da318ae16" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33613 } } }
I20260808 05:02:23.942111 6082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:23.942039 6081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58a400e59a204ce692bbcf8da318ae16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a400e59a204ce692bbcf8da318ae16" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 33613 } } }
I20260808 05:02:23.942330 6081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:23.942368 6087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:23.942899 6087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:23.944662 6087 catalog_manager.cc:1347] Generated new cluster ID: d1904df3d90c43d19878af3f7653a893
I20260808 05:02:23.944747 6087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:23.946862 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:23.941060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36898 (local address 127.3.174.190:33613)
0808 05:02:23.941341 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:23.941351 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:23.941552 (+ 201us) server_negotiation.cc:407] Connection header received
0808 05:02:23.941694 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:23.941703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:23.941779 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:23.942185 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:23.942320 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.943707 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.944745 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:23.945378 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:23.945508 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:23.946154 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:23.946158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:23.946159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:23.946376 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:23.946421 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:23.946437 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:23.946508 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:23.946565 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 05:02:23.946650 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":98,"threads_started":1}
I20260808 05:02:23.975613 6087 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:23.976377 6087 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:23.980890 6087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16: Generated new TSK 0
I20260808 05:02:23.981096 6087 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:23.986413 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.129:0
--local_ip_for_outbound_sockets=127.3.174.129
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33613
--builtin_ntp_servers=127.3.174.148:46861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:24.071128 6113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:24.071344 6113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:24.071373 6113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:24.072510 6113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:24.072547 6113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:24.072579 6113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.129
I20260808 05:02:24.073942 6113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:46861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.174.129
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.129
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:24.074167 6113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:24.074442 6113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:24.079326 6120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:24.079550 6122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:24.079592 6113 server_base.cc:1034] running on GCE node
W20260808 05:02:24.079769 6124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:24.080070 6113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:24.080322 6113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:24.081458 6113 hybrid_clock.cc:648] HybridClock initialized: now 1786165344081440 us; error 29 us; skew 500 ppm
I20260808 05:02:24.082654 6113 webserver.cc:460] Webserver started at http://127.3.174.129:36617/ using document root <none> and password file <none>
I20260808 05:02:24.082906 6113 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:24.082957 6113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:24.083088 6113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:24.084144 6113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/data/instance:
uuid: "4acf6d7681f145df8aedd0faf7fc3302"
format_stamp: "Formatted at 2026-08-08 05:02:24 on dist-test-slave-gjw7"
server_key: "c6ae3867682fc588d1c88f3b02a63a40"
server_key_iv: "df4da6dda8779c907a9168d28dfa6dfe"
server_key_version: "encryptionkey@0"
I20260808 05:02:24.084483 6113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance:
uuid: "4acf6d7681f145df8aedd0faf7fc3302"
format_stamp: "Formatted at 2026-08-08 05:02:24 on dist-test-slave-gjw7"
server_key: "c6ae3867682fc588d1c88f3b02a63a40"
server_key_iv: "df4da6dda8779c907a9168d28dfa6dfe"
server_key_version: "encryptionkey@0"
I20260808 05:02:24.085911 6113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:24.086565 6139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:24.086716 6113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:24.086828 6113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/wal
uuid: "4acf6d7681f145df8aedd0faf7fc3302"
format_stamp: "Formatted at 2026-08-08 05:02:24 on dist-test-slave-gjw7"
server_key: "c6ae3867682fc588d1c88f3b02a63a40"
server_key_iv: "df4da6dda8779c907a9168d28dfa6dfe"
server_key_version: "encryptionkey@0"
I20260808 05:02:24.086896 6113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:24.098639 6113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:24.098920 6113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:24.099045 6113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:24.099267 6113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:24.099584 6113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:24.099618 6113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:24.099647 6113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:24.099665 6113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:24.105232 6113 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.129:36443
I20260808 05:02:24.105326 6314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.129:36443 every 8 connection(s)
I20260808 05:02:24.105600 6113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/data/info.pb
I20260808 05:02:24.110585 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.106303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:38535 (local address 127.3.174.190:33613)
0808 05:02:24.106379 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.106381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.106583 (+ 202us) server_negotiation.cc:407] Connection header received
0808 05:02:24.107119 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.107120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.107136 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.107187 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:24.107871 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.108383 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.109336 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.109547 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.109604 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.110318 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:24.110319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:24.110320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:24.110325 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:24.110342 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:24.110346 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:24.110392 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.110467 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.110503 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 05:02:24.110767 6318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.106258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33613 (local address 127.3.174.129:38535)
0808 05:02:24.106473 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:24.106507 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:24.107022 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:24.107258 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:24.107263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:24.107274 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:24.107782 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.107787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.108464 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.108465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.109230 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.109232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.109618 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.109619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.109748 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.110189 (+ 441us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 05:02:24.110190 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:24.110218 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 05:02:24.110241 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:24.110397 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:24.110403 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:24.110406 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:02:24.110466 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 05:02:24.110565 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260808 05:02:24.111235 6316 heartbeater.cc:347] Connected to a master server at 127.3.174.190:33613
I20260808 05:02:24.111320 6316 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:24.111465 6316 heartbeater.cc:511] Master 127.3.174.190:33613 requested a full tablet report, sending...
I20260808 05:02:24.111924 6011 ts_manager.cc:194] Registered new tserver with Master: 4acf6d7681f145df8aedd0faf7fc3302 (127.3.174.129:36443)
I20260808 05:02:24.112543 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 6113
I20260808 05:02:24.112653 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-0/wal/instance
I20260808 05:02:24.112672 6011 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.129:38535
I20260808 05:02:24.112861 3770 external_mini_cluster.cc:1428] Setting key ec84124d4205efa2fbe2a511288c106a
I20260808 05:02:24.113830 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.130:0
--local_ip_for_outbound_sockets=127.3.174.130
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33613
--builtin_ntp_servers=127.3.174.148:46861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 05:02:24.196398 6319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:24.196599 6319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:24.196626 6319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:24.197754 6319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:24.197790 6319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:24.197824 6319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.130
I20260808 05:02:24.199147 6319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:46861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.174.130
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.130
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:24.199389 6319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:24.199671 6319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:24.204463 6324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:24.204471 6329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:24.204471 6325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:24.204713 6319 server_base.cc:1034] running on GCE node
I20260808 05:02:24.204972 6319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:24.205202 6319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:24.206379 6319 hybrid_clock.cc:648] HybridClock initialized: now 1786165344206364 us; error 32 us; skew 500 ppm
I20260808 05:02:24.207625 6319 webserver.cc:460] Webserver started at http://127.3.174.130:34389/ using document root <none> and password file <none>
I20260808 05:02:24.207870 6319 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:24.207921 6319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:24.208048 6319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:24.209128 6319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/data/instance:
uuid: "c5c667b1f9cf419aab9794b462398884"
format_stamp: "Formatted at 2026-08-08 05:02:24 on dist-test-slave-gjw7"
server_key: "0f12c04d42fb3380dd69c7e55253fac9"
server_key_iv: "418387b7dac8f6cfb9b1c33a9dcd28f1"
server_key_version: "encryptionkey@0"
I20260808 05:02:24.209488 6319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance:
uuid: "c5c667b1f9cf419aab9794b462398884"
format_stamp: "Formatted at 2026-08-08 05:02:24 on dist-test-slave-gjw7"
server_key: "0f12c04d42fb3380dd69c7e55253fac9"
server_key_iv: "418387b7dac8f6cfb9b1c33a9dcd28f1"
server_key_version: "encryptionkey@0"
I20260808 05:02:24.210686 6319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:24.211328 6338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:24.211477 6319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:24.211561 6319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/wal
uuid: "c5c667b1f9cf419aab9794b462398884"
format_stamp: "Formatted at 2026-08-08 05:02:24 on dist-test-slave-gjw7"
server_key: "0f12c04d42fb3380dd69c7e55253fac9"
server_key_iv: "418387b7dac8f6cfb9b1c33a9dcd28f1"
server_key_version: "encryptionkey@0"
I20260808 05:02:24.211623 6319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:24.230803 6319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:24.231109 6319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:24.231256 6319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:24.231474 6319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:24.231853 6319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:24.231891 6319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:24.231921 6319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:24.231945 6319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:24.237493 6319 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.130:42873
I20260808 05:02:24.237572 6506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.130:42873 every 8 connection(s)
I20260808 05:02:24.237864 6319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/data/info.pb
I20260808 05:02:24.239379 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 6319
I20260808 05:02:24.239486 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/wal/instance
I20260808 05:02:24.239666 3770 external_mini_cluster.cc:1428] Setting key 2538ea6768d119aaf743edcf7879d0e3
I20260808 05:02:24.240638 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.174.131:0
--local_ip_for_outbound_sockets=127.3.174.131
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33613
--builtin_ntp_servers=127.3.174.148:46861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 05:02:24.243208 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.238540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.130:35431 (local address 127.3.174.190:33613)
0808 05:02:24.238625 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.238628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.238817 (+ 189us) server_negotiation.cc:407] Connection header received
0808 05:02:24.239341 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.239342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.239355 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.239401 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:24.240198 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.240746 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.241831 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.242057 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.242137 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.242946 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:24.242947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:24.242948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:24.242953 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:24.242968 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:24.242972 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:24.243015 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.243086 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.243114 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 05:02:24.243410 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.238516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33613 (local address 127.3.174.130:35431)
0808 05:02:24.238699 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:24.238733 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:24.239217 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:24.239484 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:24.239488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:24.239500 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:24.240116 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.240122 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.240845 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.240846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.241735 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.241738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.242131 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.242132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.242302 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.242796 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 05:02:24.242797 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:24.242827 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 05:02:24.242850 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:24.243015 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:24.243023 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:24.243026 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:02:24.243091 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 05:02:24.243191 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260808 05:02:24.243884 6510 heartbeater.cc:347] Connected to a master server at 127.3.174.190:33613
I20260808 05:02:24.243968 6510 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:24.244139 6510 heartbeater.cc:511] Master 127.3.174.190:33613 requested a full tablet report, sending...
I20260808 05:02:24.244518 6011 ts_manager.cc:194] Registered new tserver with Master: c5c667b1f9cf419aab9794b462398884 (127.3.174.130:42873)
I20260808 05:02:24.245002 6011 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.130:35431
W20260808 05:02:24.323886 6514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:24.324070 6514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:24.324090 6514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:24.325162 6514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:24.325192 6514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:24.325215 6514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.174.131
I20260808 05:02:24.326507 6514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:46861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.174.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.174.131
--webserver_port=0
--tserver_master_addrs=127.3.174.190:33613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.174.131
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:24.326684 6514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:24.326930 6514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:24.331657 6530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:24.331702 6523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:24.331744 6514 server_base.cc:1034] running on GCE node
W20260808 05:02:24.331647 6525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:24.332041 6514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:24.332237 6514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:24.333352 6514 hybrid_clock.cc:648] HybridClock initialized: now 1786165344333340 us; error 28 us; skew 500 ppm
I20260808 05:02:24.334477 6514 webserver.cc:460] Webserver started at http://127.3.174.131:39623/ using document root <none> and password file <none>
I20260808 05:02:24.334713 6514 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:24.334758 6514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:24.334877 6514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:24.335861 6514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/data/instance:
uuid: "a71acd4a32a24235ac2938139076f568"
format_stamp: "Formatted at 2026-08-08 05:02:24 on dist-test-slave-gjw7"
server_key: "56547ae3434b2d1e1fa74cc00d1c9c4f"
server_key_iv: "4be71434dc7a22abc34a9f12ce5db70b"
server_key_version: "encryptionkey@0"
I20260808 05:02:24.336190 6514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance:
uuid: "a71acd4a32a24235ac2938139076f568"
format_stamp: "Formatted at 2026-08-08 05:02:24 on dist-test-slave-gjw7"
server_key: "56547ae3434b2d1e1fa74cc00d1c9c4f"
server_key_iv: "4be71434dc7a22abc34a9f12ce5db70b"
server_key_version: "encryptionkey@0"
I20260808 05:02:24.337453 6514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 05:02:24.338104 6544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:24.338246 6514 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:24.338325 6514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/wal
uuid: "a71acd4a32a24235ac2938139076f568"
format_stamp: "Formatted at 2026-08-08 05:02:24 on dist-test-slave-gjw7"
server_key: "56547ae3434b2d1e1fa74cc00d1c9c4f"
server_key_iv: "4be71434dc7a22abc34a9f12ce5db70b"
server_key_version: "encryptionkey@0"
I20260808 05:02:24.338390 6514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:24.344877 6514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:24.345112 6514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:24.345223 6514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:24.345445 6514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 05:02:24.345729 6514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 05:02:24.345760 6514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:24.345788 6514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 05:02:24.345813 6514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:24.351210 6514 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.131:40019
I20260808 05:02:24.351377 6723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.131:40019 every 8 connection(s)
I20260808 05:02:24.351581 6514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/data/info.pb
I20260808 05:02:24.356386 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 6514
I20260808 05:02:24.356463 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.352311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.131:56145 (local address 127.3.174.190:33613)
0808 05:02:24.352385 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.352387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.352532 (+ 145us) server_negotiation.cc:407] Connection header received
0808 05:02:24.353040 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.353041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.353054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.353104 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:24.353773 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.354275 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.355204 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.355412 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.355468 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.356179 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:24.356180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:24.356180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:24.356185 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:24.356201 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:24.356205 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:24.356250 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.356348 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.356381 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 05:02:24.356488 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/wal/instance
I20260808 05:02:24.356630 6726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.352228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:33613 (local address 127.3.174.131:56145)
0808 05:02:24.352415 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:24.352456 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:24.352936 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:24.353174 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:24.353179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:24.353190 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:24.353681 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.353685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.354360 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.354361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.355117 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.355119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.355482 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.355483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.355610 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.356047 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 05:02:24.356047 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:24.356075 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 05:02:24.356096 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:24.356255 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:24.356260 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:24.356264 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:24.356342 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 05:02:24.356443 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260808 05:02:24.356676 3770 external_mini_cluster.cc:1428] Setting key 7c7e50c969610734358d66ea2736b665
I20260808 05:02:24.357067 6724 heartbeater.cc:347] Connected to a master server at 127.3.174.190:33613
I20260808 05:02:24.357167 6724 heartbeater.cc:464] Registering TS with master...
I20260808 05:02:24.357383 6724 heartbeater.cc:511] Master 127.3.174.190:33613 requested a full tablet report, sending...
I20260808 05:02:24.357736 6011 ts_manager.cc:194] Registered new tserver with Master: a71acd4a32a24235ac2938139076f568 (127.3.174.131:40019)
I20260808 05:02:24.358088 6011 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.174.131:56145
I20260808 05:02:24.368181 3770 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 05:02:24.371974 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.369563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.3.174.190:33613)
0808 05:02:24.369630 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.369632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.369690 (+ 58us) server_negotiation.cc:407] Connection header received
0808 05:02:24.369734 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.369735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.369745 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.369796 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:24.369927 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.370378 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.371035 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.371205 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.371247 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.371788 (+ 541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:24.371789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:24.371789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:24.371794 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:24.371806 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:24.371810 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:24.371845 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.371879 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.371901 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 05:02:24.373365 6011 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 05:02:24.374857 6011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 05:02:24.381047 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.377081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:40019 (local address 127.0.0.1:36170)
0808 05:02:24.378004 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:24.378014 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:24.378058 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:24.378197 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:24.378197 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:24.378199 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:24.378231 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.378232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.379076 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.379076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.380155 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.380157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.380773 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.380775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.380851 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.380865 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 05:02:24.380924 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 05:02:24.380975 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":85,"threads_started":1}
I20260808 05:02:24.381057 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.377186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36170 (local address 127.3.174.131:40019)
0808 05:02:24.377369 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.377373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.378057 (+ 684us) server_negotiation.cc:407] Connection header received
0808 05:02:24.378118 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.378120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.378133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.378197 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:24.378280 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.378987 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.380233 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.380689 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.380751 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.380841 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.380948 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.380980 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
I20260808 05:02:24.381157 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.376906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:42873 (local address 127.0.0.1:40374)
0808 05:02:24.377668 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:24.377678 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:24.377727 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:24.377914 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:24.377915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:24.377917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:24.377953 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.377954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.379094 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.379095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.379972 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.379975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.380924 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.380925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.380989 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.380993 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:24.381038 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 05:02:24.381076 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":80,"threads_started":1}
I20260808 05:02:24.381166 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.376876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:36443 (local address 127.0.0.1:46482)
0808 05:02:24.377053 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:24.377096 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:24.377157 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:24.377587 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:24.377590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:24.377592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:24.377654 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.377657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.378884 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.378885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.380117 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.380120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.380967 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.380968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.381017 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.381020 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 05:02:24.381046 (+ 26us) client_negotiation.cc:240] Negotiation successful
0808 05:02:24.381086 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":10,"thread_start_us":69,"threads_started":1}
I20260808 05:02:24.381412 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.377195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40374 (local address 127.3.174.130:42873)
0808 05:02:24.377406 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.377410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.377789 (+ 379us) server_negotiation.cc:407] Connection header received
0808 05:02:24.377834 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.377836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.377851 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.377916 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:24.378034 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.379024 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.380049 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.380722 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.380801 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.380885 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.381270 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.381334 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":114,"threads_started":1}
I20260808 05:02:24.381534 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.376989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46482 (local address 127.3.174.129:36443)
0808 05:02:24.377442 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.377445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.377454 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:24.377486 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.377488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.377501 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.377570 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:24.377727 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.378804 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.380208 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.380883 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.381265 (+ 382us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.381386 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.381410 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.381449 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":96,"threads_started":1}
I20260808 05:02:24.382169 6215 tablet_service.cc:1467] Processing CreateTablet for tablet e5e0a06e35c542ce89d375d22ba2332c (DEFAULT_TABLE table=test-table [id=3890671ddf8f4c8aac8b29e03b4ca4a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:24.382112 6407 tablet_service.cc:1467] Processing CreateTablet for tablet e5e0a06e35c542ce89d375d22ba2332c (DEFAULT_TABLE table=test-table [id=3890671ddf8f4c8aac8b29e03b4ca4a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:24.382221 6612 tablet_service.cc:1467] Processing CreateTablet for tablet e5e0a06e35c542ce89d375d22ba2332c (DEFAULT_TABLE table=test-table [id=3890671ddf8f4c8aac8b29e03b4ca4a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 05:02:24.382520 6407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e0a06e35c542ce89d375d22ba2332c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:24.382519 6215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e0a06e35c542ce89d375d22ba2332c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:24.382530 6612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e0a06e35c542ce89d375d22ba2332c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:24.385399 6746 tablet_bootstrap.cc:492] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: Bootstrap starting.
I20260808 05:02:24.385743 6748 tablet_bootstrap.cc:492] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302: Bootstrap starting.
I20260808 05:02:24.386098 6746 tablet_bootstrap.cc:654] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:24.386411 6746 log.cc:824] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:24.386459 6748 tablet_bootstrap.cc:654] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:24.386732 6748 log.cc:824] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:24.387045 6747 tablet_bootstrap.cc:492] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: Bootstrap starting.
I20260808 05:02:24.387205 6746 tablet_bootstrap.cc:492] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: No bootstrap required, opened a new log
I20260808 05:02:24.387274 6746 ts_tablet_manager.cc:1397] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 05:02:24.387400 6748 tablet_bootstrap.cc:492] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302: No bootstrap required, opened a new log
I20260808 05:02:24.387460 6748 ts_tablet_manager.cc:1397] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 05:02:24.387660 6747 tablet_bootstrap.cc:654] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:24.387970 6747 log.cc:824] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:24.389465 6746 raft_consensus.cc:348] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } }
I20260808 05:02:24.389432 6748 raft_consensus.cc:348] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } }
I20260808 05:02:24.389621 6746 raft_consensus.cc:374] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:24.389621 6748 raft_consensus.cc:374] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:24.389664 6748 raft_consensus.cc:729] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acf6d7681f145df8aedd0faf7fc3302, State: Initialized, Role: FOLLOWER
I20260808 05:02:24.389665 6746 raft_consensus.cc:729] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a71acd4a32a24235ac2938139076f568, State: Initialized, Role: FOLLOWER
I20260808 05:02:24.389809 6746 consensus_queue.cc:260] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } }
I20260808 05:02:24.389879 6747 tablet_bootstrap.cc:492] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: No bootstrap required, opened a new log
I20260808 05:02:24.389839 6748 consensus_queue.cc:260] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } }
I20260808 05:02:24.389935 6747 ts_tablet_manager.cc:1397] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 05:02:24.390084 6746 ts_tablet_manager.cc:1428] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 05:02:24.390105 6748 ts_tablet_manager.cc:1428] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 05:02:24.390220 6316 heartbeater.cc:503] Master 127.3.174.190:33613 was elected leader, sending a full tablet report...
I20260808 05:02:24.390470 6724 heartbeater.cc:503] Master 127.3.174.190:33613 was elected leader, sending a full tablet report...
I20260808 05:02:24.391892 6747 raft_consensus.cc:348] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } }
I20260808 05:02:24.392014 6747 raft_consensus.cc:374] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:24.392032 6747 raft_consensus.cc:729] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c667b1f9cf419aab9794b462398884, State: Initialized, Role: FOLLOWER
I20260808 05:02:24.392114 6747 consensus_queue.cc:260] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } }
I20260808 05:02:24.392349 6747 ts_tablet_manager.cc:1428] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 05:02:24.392406 6510 heartbeater.cc:503] Master 127.3.174.190:33613 was elected leader, sending a full tablet report...
I20260808 05:02:24.461216 6755 raft_consensus.cc:482] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 05:02:24.461377 6755 raft_consensus.cc:504] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } }
I20260808 05:02:24.461747 6755 leader_election.cc:288] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a71acd4a32a24235ac2938139076f568 (127.3.174.131:40019), c5c667b1f9cf419aab9794b462398884 (127.3.174.130:42873)
I20260808 05:02:24.464445 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.462004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:48699 (local address 127.3.174.131:40019)
0808 05:02:24.462051 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.462054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.462064 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:24.462097 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.462098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.462112 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.462161 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:24.462348 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.462856 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.463771 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.464133 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.464194 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.464203 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.464336 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.464372 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 05:02:24.464449 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.462028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.174.129:51035 (local address 127.3.174.130:42873)
0808 05:02:24.462069 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.462071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.462145 (+ 74us) server_negotiation.cc:407] Connection header received
0808 05:02:24.462212 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.462212 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.462221 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.462269 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 05:02:24.462408 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.462869 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.463787 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.464135 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.464194 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.464203 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.464344 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.464383 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 05:02:24.464454 6318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.461911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.131:40019 (local address 127.3.174.129:48699)
0808 05:02:24.461967 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:24.461981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:24.462036 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:24.462181 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:24.462182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:24.462184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:24.462261 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.462263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.462926 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.462927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.463687 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.463689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.464204 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.464205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.464269 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.464271 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 05:02:24.464321 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 05:02:24.464369 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 05:02:24.464468 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.461921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.130:42873 (local address 127.3.174.129:51035)
0808 05:02:24.462081 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:24.462089 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:24.462149 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:24.462277 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:24.462278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:24.462279 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 05:02:24.462348 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.462350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.462940 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.462941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.463696 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:24.463698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.464207 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.464208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:24.464280 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.464284 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 05:02:24.464332 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 05:02:24.464374 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":16,"thread_start_us":86,"threads_started":1}
I20260808 05:02:24.464816 6442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e0a06e35c542ce89d375d22ba2332c" candidate_uuid: "4acf6d7681f145df8aedd0faf7fc3302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c667b1f9cf419aab9794b462398884" is_pre_election: true
I20260808 05:02:24.464884 6653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e0a06e35c542ce89d375d22ba2332c" candidate_uuid: "4acf6d7681f145df8aedd0faf7fc3302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a71acd4a32a24235ac2938139076f568" is_pre_election: true
I20260808 05:02:24.464956 6442 raft_consensus.cc:2454] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4acf6d7681f145df8aedd0faf7fc3302 in term 0.
I20260808 05:02:24.465031 6653 raft_consensus.cc:2454] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4acf6d7681f145df8aedd0faf7fc3302 in term 0.
I20260808 05:02:24.465189 6140 leader_election.cc:302] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acf6d7681f145df8aedd0faf7fc3302, c5c667b1f9cf419aab9794b462398884; no voters:
I20260808 05:02:24.465337 6755 raft_consensus.cc:2781] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 05:02:24.465394 6755 raft_consensus.cc:482] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 05:02:24.465415 6755 raft_consensus.cc:3037] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:24.466096 6755 raft_consensus.cc:504] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } }
I20260808 05:02:24.466215 6755 leader_election.cc:288] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [CANDIDATE]: Term 1 election: Requested vote from peers a71acd4a32a24235ac2938139076f568 (127.3.174.131:40019), c5c667b1f9cf419aab9794b462398884 (127.3.174.130:42873)
I20260808 05:02:24.466351 6442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e0a06e35c542ce89d375d22ba2332c" candidate_uuid: "4acf6d7681f145df8aedd0faf7fc3302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c667b1f9cf419aab9794b462398884"
I20260808 05:02:24.466375 6653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e0a06e35c542ce89d375d22ba2332c" candidate_uuid: "4acf6d7681f145df8aedd0faf7fc3302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a71acd4a32a24235ac2938139076f568"
I20260808 05:02:24.466434 6653 raft_consensus.cc:3037] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:24.466424 6442 raft_consensus.cc:3037] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:24.467142 6442 raft_consensus.cc:2454] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acf6d7681f145df8aedd0faf7fc3302 in term 1.
I20260808 05:02:24.467238 6653 raft_consensus.cc:2454] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acf6d7681f145df8aedd0faf7fc3302 in term 1.
I20260808 05:02:24.467288 6140 leader_election.cc:302] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acf6d7681f145df8aedd0faf7fc3302, c5c667b1f9cf419aab9794b462398884; no voters:
I20260808 05:02:24.467366 6755 raft_consensus.cc:2781] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:24.467502 6755 raft_consensus.cc:686] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 1 LEADER]: Becoming Leader. State: Replica: 4acf6d7681f145df8aedd0faf7fc3302, State: Running, Role: LEADER
I20260808 05:02:24.467599 6755 consensus_queue.cc:237] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } }
I20260808 05:02:24.468410 6011 catalog_manager.cc:5495] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4acf6d7681f145df8aedd0faf7fc3302 (127.3.174.129). New cstate: current_term: 1 leader_uuid: "4acf6d7681f145df8aedd0faf7fc3302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acf6d7681f145df8aedd0faf7fc3302" member_type: VOTER last_known_addr { host: "127.3.174.129" port: 36443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 } health_report { overall_health: UNKNOWN } } }
W20260808 05:02:24.488366 6511 tablet.cc:2369] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:02:24.494279 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.492274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46488 (local address 127.3.174.129:36443)
0808 05:02:24.492328 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:24.492330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:24.492337 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:02:24.492375 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:24.492376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:24.492396 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:24.492450 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:24.492544 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.492997 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.493685 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:24.493869 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:24.493913 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:24.494009 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:24.494104 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:24.494158 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:24.494170 (+ 12us) server_negotiation.cc:299] Negotiation successful
0808 05:02:24.494186 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 05:02:24.495985 6442 raft_consensus.cc:1261] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [term 1 FOLLOWER]: Refusing update from remote peer 4acf6d7681f145df8aedd0faf7fc3302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:24.495986 6653 raft_consensus.cc:1261] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [term 1 FOLLOWER]: Refusing update from remote peer 4acf6d7681f145df8aedd0faf7fc3302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:24.496312 6755 consensus_queue.cc:1035] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5c667b1f9cf419aab9794b462398884" member_type: VOTER last_known_addr { host: "127.3.174.130" port: 42873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:24.496456 6758 consensus_queue.cc:1035] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a71acd4a32a24235ac2938139076f568" member_type: VOTER last_known_addr { host: "127.3.174.131" port: 40019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:24.498550 6761 mvcc.cc:205] Tried to move back new op lower bound from 7316133251052941312 to 7316133250939830272. Current Snapshot: MvccSnapshot[applied={T|T < 7316133251052941312}]
I20260808 05:02:24.499248 6763 mvcc.cc:205] Tried to move back new op lower bound from 7316133251052941312 to 7316133250939830272. Current Snapshot: MvccSnapshot[applied={T|T < 7316133251052941312}]
I20260808 05:02:24.499562 6762 mvcc.cc:205] Tried to move back new op lower bound from 7316133251052941312 to 7316133250939830272. Current Snapshot: MvccSnapshot[applied={T|T < 7316133251052941312}]
I20260808 05:02:24.500808 6010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:24.500888 6010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 05:02:24.501578 6010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58a400e59a204ce692bbcf8da318ae16: Sending DeleteTablet for 3 replicas of tablet e5e0a06e35c542ce89d375d22ba2332c
I20260808 05:02:24.501857 6215 tablet_service.cc:1514] Processing DeleteTablet for tablet e5e0a06e35c542ce89d375d22ba2332c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:24 UTC) from {username='slave'} at 127.0.0.1:46482
I20260808 05:02:24.501915 6407 tablet_service.cc:1514] Processing DeleteTablet for tablet e5e0a06e35c542ce89d375d22ba2332c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:24 UTC) from {username='slave'} at 127.0.0.1:40374
I20260808 05:02:24.501901 6612 tablet_service.cc:1514] Processing DeleteTablet for tablet e5e0a06e35c542ce89d375d22ba2332c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 05:02:24 UTC) from {username='slave'} at 127.0.0.1:36170
I20260808 05:02:24.502115 6779 tablet_replica.cc:330] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302: stopping tablet replica
I20260808 05:02:24.502213 6779 raft_consensus.cc:2229] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 1 LEADER]: Raft consensus shutting down.
I20260808 05:02:24.502301 6779 raft_consensus.cc:2258] T e5e0a06e35c542ce89d375d22ba2332c P 4acf6d7681f145df8aedd0faf7fc3302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:24.502490 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 6113
I20260808 05:02:24.502549 6780 tablet_replica.cc:330] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: stopping tablet replica
I20260808 05:02:24.502626 6780 raft_consensus.cc:2229] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:02:24.502708 6780 raft_consensus.cc:2258] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:24.503062 6782 tablet_replica.cc:330] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: stopping tablet replica
I20260808 05:02:24.503160 6782 raft_consensus.cc:2229] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 05:02:24.503234 6780 ts_tablet_manager.cc:1905] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:24.503257 6782 raft_consensus.cc:2258] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 05:02:24.504845 6780 ts_tablet_manager.cc:1918] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:24.504900 6780 log.cc:1196] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-2/wal/wals/e5e0a06e35c542ce89d375d22ba2332c
I20260808 05:02:24.505105 6780 ts_tablet_manager.cc:1939] T e5e0a06e35c542ce89d375d22ba2332c P a71acd4a32a24235ac2938139076f568: Deleting consensus metadata
I20260808 05:02:24.505549 5987 catalog_manager.cc:4841] TS a71acd4a32a24235ac2938139076f568 (127.3.174.131:40019): tablet e5e0a06e35c542ce89d375d22ba2332c (table test-table [id=3890671ddf8f4c8aac8b29e03b4ca4a3]) successfully deleted
I20260808 05:02:24.511766 6782 ts_tablet_manager.cc:1905] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 05:02:24.513036 6782 ts_tablet_manager.cc:1918] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 05:02:24.513108 6782 log.cc:1196] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: Deleting WAL directory at /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786165096473501-3770-0/minicluster-data/ts-1/wal/wals/e5e0a06e35c542ce89d375d22ba2332c
I20260808 05:02:24.513361 6782 ts_tablet_manager.cc:1939] T e5e0a06e35c542ce89d375d22ba2332c P c5c667b1f9cf419aab9794b462398884: Deleting consensus metadata
I20260808 05:02:24.513741 5986 catalog_manager.cc:4841] TS c5c667b1f9cf419aab9794b462398884 (127.3.174.130:42873): tablet e5e0a06e35c542ce89d375d22ba2332c (table test-table [id=3890671ddf8f4c8aac8b29e03b4ca4a3]) successfully deleted
W20260808 05:02:24.515923 5987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.174.129:36443 (error 108)
I20260808 05:02:24.516383 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:24.516249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.129:36443 (local address 127.0.0.1:46494)
0808 05:02:24.516303 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:24.516349 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.129:36443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 05:02:24.516436 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 6319
W20260808 05:02:24.516500 5987 catalog_manager.cc:4584] TS 4acf6d7681f145df8aedd0faf7fc3302 (127.3.174.129:36443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5e0a06e35c542ce89d375d22ba2332c: Network error: Client connection negotiation failed: client connection to 127.3.174.129:36443: connect: Connection refused (error 111)
I20260808 05:02:24.525904 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 6514
I20260808 05:02:24.544440 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 5956
2026-08-08T05:02:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (759 ms)
[----------] 6 tests from EncryptionPolicyTest (9143 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:24 dist-test-slave-gjw7 krb5kdc[6788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:24 dist-test-slave-gjw7 krb5kdc[6788](info): set up 2 sockets
Aug 08 05:02:24 dist-test-slave-gjw7 krb5kdc[6788](info): commencing operation
krb5kdc: starting...
W20260808 05:02:25.709903 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:25 dist-test-slave-gjw7 krb5kdc[6788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:25.824422 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32867
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:25.909125 6814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:25.909379 6814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:25.909413 6814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:25.910563 6814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:25.910602 6814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:25.910621 6814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:25.910637 6814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:25.910662 6814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:25.912115 6814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:25.912356 6814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:25.912552 6814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:25.917424 6834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:25.917433 6826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:25.917434 6831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:25.917583 6814 server_base.cc:1034] running on GCE node
I20260808 05:02:25.917939 6814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:25.918243 6814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:25.919410 6814 hybrid_clock.cc:648] HybridClock initialized: now 1786165345919378 us; error 43 us; skew 500 ppm
Aug 08 05:02:25 dist-test-slave-gjw7 krb5kdc[6788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165345, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:25.926715 6814 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:25.927150 6814 webserver.cc:460] Webserver started at http://127.0.0.1:43907/ using document root <none> and password file <none>
I20260808 05:02:25.927376 6814 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:25.927426 6814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:25.927553 6814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:25.928632 6814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "188de10afb364357bbce6c8c12e8e59b"
format_stamp: "Formatted at 2026-08-08 05:02:25 on dist-test-slave-gjw7"
server_key: "b678f0d01251a11307b512afbaaae28c"
server_key_iv: "6f30aa65b96b9891e0a2e420c7611a72"
server_key_version: "encryptionkey@0"
I20260808 05:02:25.928972 6814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "188de10afb364357bbce6c8c12e8e59b"
format_stamp: "Formatted at 2026-08-08 05:02:25 on dist-test-slave-gjw7"
server_key: "b678f0d01251a11307b512afbaaae28c"
server_key_iv: "6f30aa65b96b9891e0a2e420c7611a72"
server_key_version: "encryptionkey@0"
I20260808 05:02:25.930312 6814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:02:25.930994 6857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:25.931147 6814 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:25.931237 6814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "188de10afb364357bbce6c8c12e8e59b"
format_stamp: "Formatted at 2026-08-08 05:02:25 on dist-test-slave-gjw7"
server_key: "b678f0d01251a11307b512afbaaae28c"
server_key_iv: "6f30aa65b96b9891e0a2e420c7611a72"
server_key_version: "encryptionkey@0"
I20260808 05:02:25.931314 6814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:25.954311 6814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:25.956444 6814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:25.956598 6814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:25.960587 6814 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32867
I20260808 05:02:25.960636 6947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32867 every 8 connection(s)
I20260808 05:02:25.960959 6814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:25.961985 6948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:25.964311 6948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b: Bootstrap starting.
I20260808 05:02:25.964856 6948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:25.965111 6948 log.cc:824] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:25.965763 6948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b: No bootstrap required, opened a new log
I20260808 05:02:25.967360 6948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188de10afb364357bbce6c8c12e8e59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32867 } }
I20260808 05:02:25.967495 6948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:25.967523 6948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188de10afb364357bbce6c8c12e8e59b, State: Initialized, Role: FOLLOWER
I20260808 05:02:25.967629 6948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188de10afb364357bbce6c8c12e8e59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32867 } }
I20260808 05:02:25.967677 6948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:25.967697 6948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:25.967722 6948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:25.968185 6948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188de10afb364357bbce6c8c12e8e59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32867 } }
I20260808 05:02:25.968276 6948 leader_election.cc:302] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 188de10afb364357bbce6c8c12e8e59b; no voters:
I20260808 05:02:25.968420 6948 leader_election.cc:288] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:25.968469 6951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:25.968623 6951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [term 1 LEADER]: Becoming Leader. State: Replica: 188de10afb364357bbce6c8c12e8e59b, State: Running, Role: LEADER
I20260808 05:02:25.968640 6948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:25.968719 6951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188de10afb364357bbce6c8c12e8e59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32867 } }
I20260808 05:02:25.969107 6955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 188de10afb364357bbce6c8c12e8e59b. Latest consensus state: current_term: 1 leader_uuid: "188de10afb364357bbce6c8c12e8e59b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188de10afb364357bbce6c8c12e8e59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32867 } } }
I20260808 05:02:25.969189 6955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:25.969348 6952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "188de10afb364357bbce6c8c12e8e59b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188de10afb364357bbce6c8c12e8e59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32867 } } }
I20260808 05:02:25.969525 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 6814
I20260808 05:02:25.969558 6952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:25.969633 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:25.969846 3770 external_mini_cluster.cc:1428] Setting key 9c52dafa387b8b392d9f38859080c8a6
I20260808 05:02:25.969579 6968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:25.970364 6968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:25.972133 6968 catalog_manager.cc:1347] Generated new cluster ID: f75dfa7af7cd446d9638a77a0cc6fed1
I20260808 05:02:25.972184 6968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 05:02:25 dist-test-slave-gjw7 krb5kdc[6788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 05:02:25.979784 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:25.970492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51405 (local address 127.0.0.1:32867)
0808 05:02:25.970650 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:25.970659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:25.970695 (+ 36us) server_negotiation.cc:407] Connection header received
0808 05:02:25.970796 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:25.970802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:25.970845 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:25.970956 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:25.971761 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:25.972811 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:25.973610 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:25.974025 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:25.974118 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:25.976272 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:25.976276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:25.976277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:25.976289 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:25.978262 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:25.978628 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:25.978632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:25.978632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:25.978723 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:25.978928 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:25.978929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:25.978930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:25.979154 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:25.979315 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:25.979435 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 05:02:25.979578 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260808 05:02:25.980698 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:25.987361 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:25.981988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39173 (local address 127.0.0.1:32867)
0808 05:02:25.982073 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:25.982075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:25.982124 (+ 49us) server_negotiation.cc:407] Connection header received
0808 05:02:25.982182 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:25.982182 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:25.982191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:25.982233 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:25.982705 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:25.983185 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:25.983821 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:25.984043 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:25.984095 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:25.985132 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:25.985133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:25.985134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:25.985140 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:25.986334 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:25.986657 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:25.986658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:25.986658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:25.986711 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:25.986898 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:25.986899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:25.986899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:25.987003 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:25.987087 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:25.987221 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 05:02:25.987279 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 05:02:25.995836 6968 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:25.996548 6968 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:26.005424 6968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 188de10afb364357bbce6c8c12e8e59b: Generated new TSK 0
I20260808 05:02:26.005618 6968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:26.015424 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 6814
W20260808 05:02:26.026978 6785 connection.cc:535] client connection to 127.0.0.1:32867 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32867): Connection reset by peer (error 104)
2026-08-08T05:02:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:26 dist-test-slave-gjw7 krb5kdc[7003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:26 dist-test-slave-gjw7 krb5kdc[7003](info): set up 2 sockets
Aug 08 05:02:26 dist-test-slave-gjw7 krb5kdc[7003](info): commencing operation
krb5kdc: starting...
W20260808 05:02:27.224885 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:27 dist-test-slave-gjw7 krb5kdc[7003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:27.347890 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:27.432353 7022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:27.432566 7022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:27.432590 7022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:27.433743 7022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:27.433786 7022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:27.433801 7022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:27.433815 7022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:27.433837 7022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:27.435300 7022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:27.435511 7022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:27.435693 7022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:27.440423 7034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:27.440495 7022 server_base.cc:1034] running on GCE node
W20260808 05:02:27.440466 7035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:27.440415 7038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:27.440899 7022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:27.441166 7022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:27.442304 7022 hybrid_clock.cc:648] HybridClock initialized: now 1786165347442286 us; error 38 us; skew 500 ppm
Aug 08 05:02:27 dist-test-slave-gjw7 krb5kdc[7003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165347, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:27.444659 7022 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:27.445075 7022 webserver.cc:460] Webserver started at http://127.0.0.1:44165/ using document root <none> and password file <none>
I20260808 05:02:27.445274 7022 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:27.445350 7022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:27.445474 7022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:27.446485 7022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "f5ca468325e240cc91d812f84e3dd477"
format_stamp: "Formatted at 2026-08-08 05:02:27 on dist-test-slave-gjw7"
server_key: "86de37773073203093d897c855897257"
server_key_iv: "f6cf9492e94c9b6ba573b4da4369fec0"
server_key_version: "encryptionkey@0"
I20260808 05:02:27.446816 7022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "f5ca468325e240cc91d812f84e3dd477"
format_stamp: "Formatted at 2026-08-08 05:02:27 on dist-test-slave-gjw7"
server_key: "86de37773073203093d897c855897257"
server_key_iv: "f6cf9492e94c9b6ba573b4da4369fec0"
server_key_version: "encryptionkey@0"
I20260808 05:02:27.448016 7022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 05:02:27.448625 7051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:27.448766 7022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:27.448868 7022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "f5ca468325e240cc91d812f84e3dd477"
format_stamp: "Formatted at 2026-08-08 05:02:27 on dist-test-slave-gjw7"
server_key: "86de37773073203093d897c855897257"
server_key_iv: "f6cf9492e94c9b6ba573b4da4369fec0"
server_key_version: "encryptionkey@0"
I20260808 05:02:27.448930 7022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:27.470095 7022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:27.471871 7022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:27.472005 7022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:27.475899 7022 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41733
I20260808 05:02:27.475947 7135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41733 every 8 connection(s)
I20260808 05:02:27.476276 7022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:27.477226 7136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:27.479583 7136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477: Bootstrap starting.
I20260808 05:02:27.480154 7136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:27.480436 7136 log.cc:824] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:27.481108 7136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477: No bootstrap required, opened a new log
I20260808 05:02:27.482805 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 7022
I20260808 05:02:27.482712 7136 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ca468325e240cc91d812f84e3dd477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41733 } }
I20260808 05:02:27.482859 7136 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:27.482900 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:27.482892 7136 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ca468325e240cc91d812f84e3dd477, State: Initialized, Role: FOLLOWER
I20260808 05:02:27.483024 7136 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ca468325e240cc91d812f84e3dd477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41733 } }
I20260808 05:02:27.483078 3770 external_mini_cluster.cc:1428] Setting key acf41d5d1a590a1ab9f2bde27fa3587d
I20260808 05:02:27.483104 7136 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:27.483139 7136 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:27.483186 7136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:27.484200 7136 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ca468325e240cc91d812f84e3dd477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41733 } }
I20260808 05:02:27.484313 7136 leader_election.cc:302] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ca468325e240cc91d812f84e3dd477; no voters:
I20260808 05:02:27.484476 7136 leader_election.cc:288] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:27.484524 7149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:27.484654 7149 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [term 1 LEADER]: Becoming Leader. State: Replica: f5ca468325e240cc91d812f84e3dd477, State: Running, Role: LEADER
I20260808 05:02:27.484714 7136 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:27.484789 7149 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ca468325e240cc91d812f84e3dd477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41733 } }
I20260808 05:02:27.485446 7152 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5ca468325e240cc91d812f84e3dd477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ca468325e240cc91d812f84e3dd477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41733 } } }
I20260808 05:02:27.485562 7152 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:27.485752 7152 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ca468325e240cc91d812f84e3dd477. Latest consensus state: current_term: 1 leader_uuid: "f5ca468325e240cc91d812f84e3dd477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ca468325e240cc91d812f84e3dd477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41733 } } }
I20260808 05:02:27.485823 7152 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477 [sys.catalog]: This master's current role is: LEADER
W20260808 05:02:27.486685 7185 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:02:27.486788 7185 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:02:27.486914 7166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:27.487107 7166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:27.488844 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:27.483515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43185 (local address 127.0.0.1:41733)
0808 05:02:27.483735 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:27.483746 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:27.483786 (+ 40us) server_negotiation.cc:407] Connection header received
0808 05:02:27.483918 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:27.483926 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:27.483994 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:27.484139 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:27.484327 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:27.485735 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:27.486750 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:27.487286 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:27.487385 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:27.488085 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:27.488089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:27.488090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:27.488244 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:27.488282 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:27.488293 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:27.488474 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:27.488576 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 05:02:27.488626 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
I20260808 05:02:27.489853 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:27.494458 7166 catalog_manager.cc:1347] Generated new cluster ID: 24bfa2fa75814e078a35c6b630600b93
I20260808 05:02:27.494513 7166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:27.497422 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:27.491394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39445 (local address 127.0.0.1:41733)
0808 05:02:27.491449 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:27.491450 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:27.493948 (+ 2498us) server_negotiation.cc:407] Connection header received
0808 05:02:27.494013 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:27.494014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:27.494025 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:27.494089 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:27.494228 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:27.495087 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:27.496130 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:27.496438 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:27.496494 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:27.497142 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:27.497143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:27.497144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:27.497160 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:27.497173 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:27.497178 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:27.497228 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:27.497337 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 05:02:27.497348 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 05:02:27.518885 7166 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:27.519479 7166 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:27.528659 7166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5ca468325e240cc91d812f84e3dd477: Generated new TSK 0
I20260808 05:02:27.528877 7166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:27.539628 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 7022
W20260808 05:02:27.551582 6999 connection.cc:535] client connection to 127.0.0.1:41733 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41733): Connection reset by peer (error 104)
2026-08-08T05:02:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:27 dist-test-slave-gjw7 krb5kdc[7201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:27 dist-test-slave-gjw7 krb5kdc[7201](info): set up 2 sockets
Aug 08 05:02:27 dist-test-slave-gjw7 krb5kdc[7201](info): commencing operation
krb5kdc: starting...
W20260808 05:02:28.733237 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:28 dist-test-slave-gjw7 krb5kdc[7201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:28.847538 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:28.929128 7219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:28.929358 7219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:28.929380 7219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:28.930477 7219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:28.930511 7219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:28.930524 7219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:28.930536 7219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:28.930548 7219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:28.931968 7219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45993
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:28.932158 7219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:28.932343 7219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:28.937000 7232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:28.937019 7219 server_base.cc:1034] running on GCE node
W20260808 05:02:28.936980 7233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:28.939205 7235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:28.939494 7219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:28.939774 7219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:28.940919 7219 hybrid_clock.cc:648] HybridClock initialized: now 1786165348940903 us; error 35 us; skew 500 ppm
Aug 08 05:02:28 dist-test-slave-gjw7 krb5kdc[7201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165348, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:28.943390 7219 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:28.943817 7219 webserver.cc:460] Webserver started at http://127.0.0.1:32879/ using document root <none> and password file <none>
I20260808 05:02:28.944016 7219 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:28.944065 7219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:28.944190 7219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:28.945238 7219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "e7f76a63aafa4d8ca35be9bdbe591274"
format_stamp: "Formatted at 2026-08-08 05:02:28 on dist-test-slave-gjw7"
server_key: "573dfd0f399372b5230f526783d067d0"
server_key_iv: "600b5b385af06c499316e5cef48f0278"
server_key_version: "encryptionkey@0"
I20260808 05:02:28.945631 7219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "e7f76a63aafa4d8ca35be9bdbe591274"
format_stamp: "Formatted at 2026-08-08 05:02:28 on dist-test-slave-gjw7"
server_key: "573dfd0f399372b5230f526783d067d0"
server_key_iv: "600b5b385af06c499316e5cef48f0278"
server_key_version: "encryptionkey@0"
I20260808 05:02:28.946870 7219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:28.947544 7245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:28.947691 7219 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:28.947775 7219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "e7f76a63aafa4d8ca35be9bdbe591274"
format_stamp: "Formatted at 2026-08-08 05:02:28 on dist-test-slave-gjw7"
server_key: "573dfd0f399372b5230f526783d067d0"
server_key_iv: "600b5b385af06c499316e5cef48f0278"
server_key_version: "encryptionkey@0"
I20260808 05:02:28.947856 7219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:28.951119 7219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:28.952881 7219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:28.953008 7219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:28.957002 7219 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45993
I20260808 05:02:28.957031 7344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45993 every 8 connection(s)
I20260808 05:02:28.957368 7219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:28.958339 7346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:28.960608 7346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274: Bootstrap starting.
I20260808 05:02:28.961160 7346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:28.961480 7346 log.cc:824] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:28.961831 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 7219
I20260808 05:02:28.961930 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:28.962116 3770 external_mini_cluster.cc:1428] Setting key 7d17d72513b9589f0925784da9fa4dfa
I20260808 05:02:28.962177 7346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274: No bootstrap required, opened a new log
I20260808 05:02:28.964323 7346 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f76a63aafa4d8ca35be9bdbe591274" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45993 } }
I20260808 05:02:28.964493 7346 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:28.964560 7346 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7f76a63aafa4d8ca35be9bdbe591274, State: Initialized, Role: FOLLOWER
I20260808 05:02:28.964700 7346 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f76a63aafa4d8ca35be9bdbe591274" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45993 } }
I20260808 05:02:28.964804 7346 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:28.964809 7353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:28.962614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56413 (local address 127.0.0.1:45993)
0808 05:02:28.962804 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:28.962816 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:28.962853 (+ 37us) server_negotiation.cc:407] Connection header received
0808 05:02:28.963485 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:28.963495 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:28.963531 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:28.963673 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:28.964232 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:28.964235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:28.964236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:28.964240 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:28.964285 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:28.964309 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:28.964383 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:28.964416 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 05:02:28.964455 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260808 05:02:28.964855 7346 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:28.964900 7346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:28.965632 7346 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f76a63aafa4d8ca35be9bdbe591274" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45993 } }
I20260808 05:02:28.965764 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:28.965754 7346 leader_election.cc:302] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7f76a63aafa4d8ca35be9bdbe591274; no voters:
I20260808 05:02:28.965922 7346 leader_election.cc:288] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:28.965951 7359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:28.966089 7359 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [term 1 LEADER]: Becoming Leader. State: Replica: e7f76a63aafa4d8ca35be9bdbe591274, State: Running, Role: LEADER
I20260808 05:02:28.966141 7346 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:28.966274 7359 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f76a63aafa4d8ca35be9bdbe591274" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45993 } }
I20260808 05:02:28.966612 7360 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7f76a63aafa4d8ca35be9bdbe591274" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f76a63aafa4d8ca35be9bdbe591274" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45993 } } }
I20260808 05:02:28.966696 7360 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:28.966926 7361 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7f76a63aafa4d8ca35be9bdbe591274. Latest consensus state: current_term: 1 leader_uuid: "e7f76a63aafa4d8ca35be9bdbe591274" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f76a63aafa4d8ca35be9bdbe591274" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45993 } } }
I20260808 05:02:28.967038 7361 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:28.967206 7368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:28.967762 7368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:28.969254 7368 catalog_manager.cc:1347] Generated new cluster ID: 58608647c2504bbcbf3bd060be1df19a
I20260808 05:02:28.969368 7368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:28.969782 7353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:28.968614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53241 (local address 127.0.0.1:45993)
0808 05:02:28.968642 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:28.968644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:28.968774 (+ 130us) server_negotiation.cc:407] Connection header received
0808 05:02:28.968849 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:28.968850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:28.968855 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:28.968901 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:28.969566 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:28.969567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:28.969567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:28.969572 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:28.969587 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:28.969591 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:28.969649 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:28.969703 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 05:02:28.969715 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260808 05:02:28.990909 7368 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:28.991072 7368 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:28.995091 7368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7f76a63aafa4d8ca35be9bdbe591274: Generated new TSK 0
I20260808 05:02:28.995242 7368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:29.002930 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 7219
2026-08-08T05:02:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:29 dist-test-slave-gjw7 krb5kdc[7403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:29 dist-test-slave-gjw7 krb5kdc[7403](info): set up 2 sockets
Aug 08 05:02:29 dist-test-slave-gjw7 krb5kdc[7403](info): commencing operation
krb5kdc: starting...
W20260808 05:02:30.192616 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:30 dist-test-slave-gjw7 krb5kdc[7403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:30.309928 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36043
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:30.393368 7428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:30.393581 7428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:30.393610 7428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:30.394811 7428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:30.394857 7428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:30.394876 7428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:30.394892 7428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:30.394906 7428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:30.396411 7428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:30.396667 7428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:30.396873 7428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:30.401639 7439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:30.401669 7436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:30.401649 7434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:30.401870 7428 server_base.cc:1034] running on GCE node
I20260808 05:02:30.402124 7428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:30.402405 7428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:30.403545 7428 hybrid_clock.cc:648] HybridClock initialized: now 1786165350403532 us; error 30 us; skew 500 ppm
Aug 08 05:02:30 dist-test-slave-gjw7 krb5kdc[7403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165350, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:30.405965 7428 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:30.406384 7428 webserver.cc:460] Webserver started at http://127.0.0.1:36945/ using document root <none> and password file <none>
I20260808 05:02:30.406584 7428 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:30.406632 7428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:30.406759 7428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:30.407807 7428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "91571f9a47dd44a692d30d5e20d64f55"
format_stamp: "Formatted at 2026-08-08 05:02:30 on dist-test-slave-gjw7"
server_key: "e0ca5b2ce9591589ffe56c39b35f79f0"
server_key_iv: "356b7ca109afbee5df06727291f145fe"
server_key_version: "encryptionkey@0"
I20260808 05:02:30.408146 7428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "91571f9a47dd44a692d30d5e20d64f55"
format_stamp: "Formatted at 2026-08-08 05:02:30 on dist-test-slave-gjw7"
server_key: "e0ca5b2ce9591589ffe56c39b35f79f0"
server_key_iv: "356b7ca109afbee5df06727291f145fe"
server_key_version: "encryptionkey@0"
I20260808 05:02:30.409399 7428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 05:02:30.410068 7449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:30.410220 7428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:30.410317 7428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "91571f9a47dd44a692d30d5e20d64f55"
format_stamp: "Formatted at 2026-08-08 05:02:30 on dist-test-slave-gjw7"
server_key: "e0ca5b2ce9591589ffe56c39b35f79f0"
server_key_iv: "356b7ca109afbee5df06727291f145fe"
server_key_version: "encryptionkey@0"
I20260808 05:02:30.410380 7428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:30.431586 7428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:30.433444 7428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:30.433586 7428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:30.437613 7428 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36043
I20260808 05:02:30.437646 7543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36043 every 8 connection(s)
I20260808 05:02:30.437994 7428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:30.438961 7546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:30.441378 7546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55: Bootstrap starting.
I20260808 05:02:30.441949 7546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:30.442220 7546 log.cc:824] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:30.442901 7546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55: No bootstrap required, opened a new log
I20260808 05:02:30.444613 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 7428
I20260808 05:02:30.444571 7546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91571f9a47dd44a692d30d5e20d64f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36043 } }
I20260808 05:02:30.444703 7546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:30.444732 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:30.444746 7546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91571f9a47dd44a692d30d5e20d64f55, State: Initialized, Role: FOLLOWER
I20260808 05:02:30.444897 7546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91571f9a47dd44a692d30d5e20d64f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36043 } }
I20260808 05:02:30.444976 7546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:30.444993 3770 external_mini_cluster.cc:1428] Setting key cae07106c3733fa3d5cf4613997553da
I20260808 05:02:30.445010 7546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:30.445053 7546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:30.445997 7546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91571f9a47dd44a692d30d5e20d64f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36043 } }
I20260808 05:02:30.446102 7546 leader_election.cc:302] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91571f9a47dd44a692d30d5e20d64f55; no voters:
I20260808 05:02:30.446269 7546 leader_election.cc:288] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:30.446290 7553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:30.446403 7553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [term 1 LEADER]: Becoming Leader. State: Replica: 91571f9a47dd44a692d30d5e20d64f55, State: Running, Role: LEADER
I20260808 05:02:30.446542 7546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:30.446547 7553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91571f9a47dd44a692d30d5e20d64f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36043 } }
I20260808 05:02:30.447849 7558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91571f9a47dd44a692d30d5e20d64f55. Latest consensus state: current_term: 1 leader_uuid: "91571f9a47dd44a692d30d5e20d64f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91571f9a47dd44a692d30d5e20d64f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36043 } } }
I20260808 05:02:30.447947 7558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:30.447908 7556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91571f9a47dd44a692d30d5e20d64f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91571f9a47dd44a692d30d5e20d64f55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36043 } } }
I20260808 05:02:30.448046 7556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:30.448177 7590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:30.448669 7590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:30.450062 7590 catalog_manager.cc:1347] Generated new cluster ID: d0be0d131cfd40119dd2b388872a9571
I20260808 05:02:30.450114 7590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 05:02:30 dist-test-slave-gjw7 krb5kdc[7403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 05:02:30.456336 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:30.445456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42071 (local address 127.0.0.1:36043)
0808 05:02:30.445664 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:30.445675 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:30.445709 (+ 34us) server_negotiation.cc:407] Connection header received
0808 05:02:30.445811 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:30.445816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:30.445868 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:30.445980 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:30.446692 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:30.448055 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:30.449105 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:30.449723 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:30.449819 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:30.452598 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:30.452603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:30.452603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:30.452617 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:30.454557 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:30.454951 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:30.454954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:30.454954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:30.455079 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:30.455299 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:30.455301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:30.455301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:30.455659 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:30.455885 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:30.455979 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 05:02:30.456127 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
I20260808 05:02:30.457263 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:30.465548 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:30.458745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60877 (local address 127.0.0.1:36043)
0808 05:02:30.458808 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:30.458810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:30.459010 (+ 200us) server_negotiation.cc:407] Connection header received
0808 05:02:30.459071 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:30.459072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:30.459082 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:30.459120 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:30.459725 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:30.460437 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:30.461444 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:30.461742 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:30.461799 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:30.463031 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:30.463032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:30.463033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:30.463039 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:30.464466 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:30.464834 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:30.464835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:30.464835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:30.464883 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:30.465050 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:30.465051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:30.465051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:30.465158 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:30.465250 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:30.465395 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 05:02:30.465461 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 05:02:30.480472 7590 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:30.481235 7590 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:30.484115 7590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91571f9a47dd44a692d30d5e20d64f55: Generated new TSK 0
I20260808 05:02:30.484289 7590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:30.491057 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 7428
2026-08-08T05:02:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:30 dist-test-slave-gjw7 krb5kdc[7615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:30 dist-test-slave-gjw7 krb5kdc[7615](info): set up 2 sockets
Aug 08 05:02:30 dist-test-slave-gjw7 krb5kdc[7615](info): commencing operation
krb5kdc: starting...
W20260808 05:02:31.673097 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:31 dist-test-slave-gjw7 krb5kdc[7615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:31.787650 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:31.870275 7643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:31.870472 7643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:31.870504 7643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:31.871649 7643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:31.871695 7643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:31.871716 7643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:31.871731 7643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:31.871747 7643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:31.873198 7643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:31.873447 7643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:31.873647 7643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:31.878181 7651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:31.878198 7650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:31.878408 7643 server_base.cc:1034] running on GCE node
W20260808 05:02:31.878189 7654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:31.878768 7643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:31.879079 7643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:31.880249 7643 hybrid_clock.cc:648] HybridClock initialized: now 1786165351880227 us; error 39 us; skew 500 ppm
Aug 08 05:02:31 dist-test-slave-gjw7 krb5kdc[7615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165351, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:31.885986 7643 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:31.886539 7643 webserver.cc:460] Webserver started at http://127.0.0.1:42329/ using document root <none> and password file <none>
I20260808 05:02:31.886726 7643 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:31.886765 7643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:31.886854 7643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:31.887833 7643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "818ade6a679c47f7b7065b488a9000bb"
format_stamp: "Formatted at 2026-08-08 05:02:31 on dist-test-slave-gjw7"
server_key: "206135c2ba3a4e17e7abeee04242c9b5"
server_key_iv: "748e97a8d30c2a71ccad6dddf72b6d7f"
server_key_version: "encryptionkey@0"
I20260808 05:02:31.888123 7643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "818ade6a679c47f7b7065b488a9000bb"
format_stamp: "Formatted at 2026-08-08 05:02:31 on dist-test-slave-gjw7"
server_key: "206135c2ba3a4e17e7abeee04242c9b5"
server_key_iv: "748e97a8d30c2a71ccad6dddf72b6d7f"
server_key_version: "encryptionkey@0"
I20260808 05:02:31.889338 7643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 05:02:31.889984 7665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:31.890113 7643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:31.890194 7643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "818ade6a679c47f7b7065b488a9000bb"
format_stamp: "Formatted at 2026-08-08 05:02:31 on dist-test-slave-gjw7"
server_key: "206135c2ba3a4e17e7abeee04242c9b5"
server_key_iv: "748e97a8d30c2a71ccad6dddf72b6d7f"
server_key_version: "encryptionkey@0"
I20260808 05:02:31.890259 7643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:31.906144 7643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:31.908131 7643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:31.908257 7643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:31.912195 7643 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45649
I20260808 05:02:31.912230 7761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45649 every 8 connection(s)
I20260808 05:02:31.912554 7643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:31.913539 7762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:31.915783 7762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb: Bootstrap starting.
I20260808 05:02:31.916360 7762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:31.916615 7762 log.cc:824] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:31.917241 7762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb: No bootstrap required, opened a new log
I20260808 05:02:31.918874 7762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818ade6a679c47f7b7065b488a9000bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45649 } }
I20260808 05:02:31.919013 7762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:31.919044 7762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818ade6a679c47f7b7065b488a9000bb, State: Initialized, Role: FOLLOWER
I20260808 05:02:31.919178 7762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818ade6a679c47f7b7065b488a9000bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45649 } }
I20260808 05:02:31.919240 7762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:31.919273 7762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:31.919318 7762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:31.919823 7762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818ade6a679c47f7b7065b488a9000bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45649 } }
I20260808 05:02:31.919915 7762 leader_election.cc:302] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 818ade6a679c47f7b7065b488a9000bb; no voters:
I20260808 05:02:31.920076 7762 leader_election.cc:288] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:31.920109 7766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:31.920264 7766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [term 1 LEADER]: Becoming Leader. State: Replica: 818ade6a679c47f7b7065b488a9000bb, State: Running, Role: LEADER
I20260808 05:02:31.920305 7762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:31.920382 7766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818ade6a679c47f7b7065b488a9000bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45649 } }
I20260808 05:02:31.920811 7768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 818ade6a679c47f7b7065b488a9000bb. Latest consensus state: current_term: 1 leader_uuid: "818ade6a679c47f7b7065b488a9000bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818ade6a679c47f7b7065b488a9000bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45649 } } }
I20260808 05:02:31.920902 7768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:31.920922 7767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "818ade6a679c47f7b7065b488a9000bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818ade6a679c47f7b7065b488a9000bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45649 } } }
I20260808 05:02:31.920971 7767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:31.921126 7782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:31.921675 7782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:31.922667 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 7643
I20260808 05:02:31.922786 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:31.922984 3770 external_mini_cluster.cc:1428] Setting key 0a4b1fe89010643dcd81c4ca6868e39f
I20260808 05:02:31.923280 7782 catalog_manager.cc:1347] Generated new cluster ID: 6289da8adf924dad98f9ee515d1d0295
I20260808 05:02:31.923334 7782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:31.930557 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:31.923543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39453 (local address 127.0.0.1:45649)
0808 05:02:31.925390 (+ 1847us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:31.925400 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:31.925432 (+ 32us) server_negotiation.cc:407] Connection header received
0808 05:02:31.925543 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:31.925549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:31.925600 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:31.925722 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:31.925966 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:31.927348 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:31.928442 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:31.929024 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:31.929117 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:31.929794 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:31.929799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:31.929799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:31.929994 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:31.930031 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:31.930046 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:31.930105 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:31.930179 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 05:02:31.930341 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1798,"thread_start_us":69,"threads_started":1}
I20260808 05:02:31.931409 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:31.936167 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:31.932793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45931 (local address 127.0.0.1:45649)
0808 05:02:31.932825 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:31.932827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:31.932939 (+ 112us) server_negotiation.cc:407] Connection header received
0808 05:02:31.933026 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:31.933027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:31.933036 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:31.933059 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:31.933220 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:31.933977 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:31.934942 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:31.935227 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:31.935258 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:31.935932 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:31.935933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:31.935933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:31.935938 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:31.935951 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:31.935957 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:31.935981 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:31.936070 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 05:02:31.936094 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 05:02:31.938254 7782 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:31.938724 7782 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:31.941524 7782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 818ade6a679c47f7b7065b488a9000bb: Generated new TSK 0
I20260808 05:02:31.941695 7782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:31.949853 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 7643
2026-08-08T05:02:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:32 dist-test-slave-gjw7 krb5kdc[7815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:32 dist-test-slave-gjw7 krb5kdc[7815](info): set up 2 sockets
Aug 08 05:02:32 dist-test-slave-gjw7 krb5kdc[7815](info): commencing operation
krb5kdc: starting...
W20260808 05:02:33.122735 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:33 dist-test-slave-gjw7 krb5kdc[7815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:33.237612 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:33.320801 7843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:33.320992 7843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:33.321013 7843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:33.322190 7843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:33.322233 7843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:33.322247 7843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:33.322278 7843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:33.322293 7843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:33.323755 7843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:33.323979 7843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:33.324182 7843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:33.328765 7852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:33.328889 7843 server_base.cc:1034] running on GCE node
W20260808 05:02:33.328773 7854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:33.328783 7851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:33.329227 7843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:33.329514 7843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:33.330654 7843 hybrid_clock.cc:648] HybridClock initialized: now 1786165353330634 us; error 34 us; skew 500 ppm
Aug 08 05:02:33 dist-test-slave-gjw7 krb5kdc[7815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165353, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:33.333046 7843 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:33.333480 7843 webserver.cc:460] Webserver started at http://127.0.0.1:44155/ using document root <none> and password file <none>
I20260808 05:02:33.333683 7843 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:33.333721 7843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:33.333814 7843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:33.334797 7843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "ca924c132fa644c987db9e0b8cfbed08"
format_stamp: "Formatted at 2026-08-08 05:02:33 on dist-test-slave-gjw7"
server_key: "cc7d33ceb4ec2f7030c9dd56f42d5a2f"
server_key_iv: "5bdd1532d625b34807b429db8162d9d8"
server_key_version: "encryptionkey@0"
I20260808 05:02:33.335081 7843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "ca924c132fa644c987db9e0b8cfbed08"
format_stamp: "Formatted at 2026-08-08 05:02:33 on dist-test-slave-gjw7"
server_key: "cc7d33ceb4ec2f7030c9dd56f42d5a2f"
server_key_iv: "5bdd1532d625b34807b429db8162d9d8"
server_key_version: "encryptionkey@0"
I20260808 05:02:33.336234 7843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:33.336889 7869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:33.337025 7843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:33.337108 7843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "ca924c132fa644c987db9e0b8cfbed08"
format_stamp: "Formatted at 2026-08-08 05:02:33 on dist-test-slave-gjw7"
server_key: "cc7d33ceb4ec2f7030c9dd56f42d5a2f"
server_key_iv: "5bdd1532d625b34807b429db8162d9d8"
server_key_version: "encryptionkey@0"
I20260808 05:02:33.337168 7843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:33.340286 7843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:33.342322 7843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:33.342444 7843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:33.346225 7843 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37227
I20260808 05:02:33.346253 7974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37227 every 8 connection(s)
I20260808 05:02:33.346570 7843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:33.347536 7975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:33.349880 7975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08: Bootstrap starting.
I20260808 05:02:33.350418 7975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:33.350683 7975 log.cc:824] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:33.351301 7975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08: No bootstrap required, opened a new log
I20260808 05:02:33.351958 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 7843
I20260808 05:02:33.352085 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:33.352299 3770 external_mini_cluster.cc:1428] Setting key e65719e49ec6055a1ae3f77cde077005
I20260808 05:02:33.353327 7975 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca924c132fa644c987db9e0b8cfbed08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } }
I20260808 05:02:33.353487 7975 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:33.353560 7975 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca924c132fa644c987db9e0b8cfbed08, State: Initialized, Role: FOLLOWER
I20260808 05:02:33.353695 7975 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca924c132fa644c987db9e0b8cfbed08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } }
I20260808 05:02:33.353780 7975 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:33.353811 7975 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:33.353850 7975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:33.354574 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:33.352785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37055 (local address 127.0.0.1:37227)
0808 05:02:33.352949 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:33.352957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:33.352982 (+ 25us) server_negotiation.cc:407] Connection header received
0808 05:02:33.353254 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:33.353260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:33.353284 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:33.353435 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:33.354114 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:33.354118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:33.354118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:33.354123 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:33.354181 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:33.354193 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:33.354245 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:33.354340 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 05:02:33.354375 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260808 05:02:33.354837 7975 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca924c132fa644c987db9e0b8cfbed08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } }
I20260808 05:02:33.354948 7975 leader_election.cc:302] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca924c132fa644c987db9e0b8cfbed08; no voters:
I20260808 05:02:33.355108 7975 leader_election.cc:288] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:33.355244 7984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:33.355358 7975 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:33.355417 7984 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [term 1 LEADER]: Becoming Leader. State: Replica: ca924c132fa644c987db9e0b8cfbed08, State: Running, Role: LEADER
I20260808 05:02:33.355469 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:33.355528 7984 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca924c132fa644c987db9e0b8cfbed08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } }
I20260808 05:02:33.355837 7984 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca924c132fa644c987db9e0b8cfbed08. Latest consensus state: current_term: 1 leader_uuid: "ca924c132fa644c987db9e0b8cfbed08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca924c132fa644c987db9e0b8cfbed08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } } }
I20260808 05:02:33.355985 7984 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:33.356230 8002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:33.356739 8002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:33.357731 7986 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca924c132fa644c987db9e0b8cfbed08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca924c132fa644c987db9e0b8cfbed08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } } }
I20260808 05:02:33.357887 7986 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:33.358072 8025 mvcc.cc:205] Tried to move back new op lower bound from 7316133287352913920 to 7316133287344586752. Current Snapshot: MvccSnapshot[applied={T|T < 7316133287352913920}]
I20260808 05:02:33.358076 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:33.357166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48215 (local address 127.0.0.1:37227)
0808 05:02:33.357196 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:33.357198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:33.357206 (+ 8us) server_negotiation.cc:407] Connection header received
0808 05:02:33.357224 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:33.357225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:33.357231 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:33.357278 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:33.357849 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:33.357850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:33.357851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:33.357855 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:33.357870 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:33.357873 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:33.357900 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:33.358006 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 05:02:33.358020 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 05:02:33.358537 8002 catalog_manager.cc:1347] Generated new cluster ID: cb874be5ace44e22a7bb8105b3aeb7e3
I20260808 05:02:33.358587 8002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:33.376559 8002 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:33.376757 8002 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:33.380975 8002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca924c132fa644c987db9e0b8cfbed08: Generated new TSK 0
I20260808 05:02:33.381140 8002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:33.386637 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 7843
2026-08-08T05:02:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:33 dist-test-slave-gjw7 krb5kdc[8031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:33 dist-test-slave-gjw7 krb5kdc[8031](info): set up 2 sockets
Aug 08 05:02:33 dist-test-slave-gjw7 krb5kdc[8031](info): commencing operation
krb5kdc: starting...
W20260808 05:02:34.569497 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:34 dist-test-slave-gjw7 krb5kdc[8031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:34.688094 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:34.772975 8048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:34.773182 8048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:34.773213 8048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:34.774403 8048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:34.774447 8048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:34.774466 8048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:34.774482 8048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:34.774499 8048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:34.776013 8048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:34.776258 8048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:34.776461 8048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:34.781147 8057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:34.781108 8055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:34.781328 8048 server_base.cc:1034] running on GCE node
W20260808 05:02:34.781103 8054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:34.781661 8048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:34.781941 8048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:34.783078 8048 hybrid_clock.cc:648] HybridClock initialized: now 1786165354783066 us; error 38 us; skew 500 ppm
Aug 08 05:02:34 dist-test-slave-gjw7 krb5kdc[8031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165354, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:34.785460 8048 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:34.785938 8048 webserver.cc:460] Webserver started at http://127.0.0.1:33095/ using document root <none> and password file <none>
I20260808 05:02:34.786167 8048 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:34.786217 8048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:34.786346 8048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:34.787381 8048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "14786700b2f149d5a456718d5825fe49"
format_stamp: "Formatted at 2026-08-08 05:02:34 on dist-test-slave-gjw7"
server_key: "e9659b8665d8eadea8fba2bd4a14455f"
server_key_iv: "fa65e7fc529a2abc725f71d16181f07e"
server_key_version: "encryptionkey@0"
I20260808 05:02:34.787726 8048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "14786700b2f149d5a456718d5825fe49"
format_stamp: "Formatted at 2026-08-08 05:02:34 on dist-test-slave-gjw7"
server_key: "e9659b8665d8eadea8fba2bd4a14455f"
server_key_iv: "fa65e7fc529a2abc725f71d16181f07e"
server_key_version: "encryptionkey@0"
I20260808 05:02:34.788967 8048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:34.789703 8073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:34.789886 8048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:34.789974 8048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "14786700b2f149d5a456718d5825fe49"
format_stamp: "Formatted at 2026-08-08 05:02:34 on dist-test-slave-gjw7"
server_key: "e9659b8665d8eadea8fba2bd4a14455f"
server_key_iv: "fa65e7fc529a2abc725f71d16181f07e"
server_key_version: "encryptionkey@0"
I20260808 05:02:34.790042 8048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:34.801617 8048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:34.803503 8048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:34.803634 8048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:34.807816 8048 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32773
I20260808 05:02:34.807853 8170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32773 every 8 connection(s)
I20260808 05:02:34.808203 8048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:34.809254 8171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:34.811774 8171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49: Bootstrap starting.
I20260808 05:02:34.812362 8171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:34.812620 8171 log.cc:824] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:34.813154 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 8048
I20260808 05:02:34.813328 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:34.813465 8171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49: No bootstrap required, opened a new log
I20260808 05:02:34.813587 3770 external_mini_cluster.cc:1428] Setting key c34fb1ac4ff2c0f482d18897603e6f75
I20260808 05:02:34.815464 8171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14786700b2f149d5a456718d5825fe49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260808 05:02:34.815641 8171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:34.815697 8171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14786700b2f149d5a456718d5825fe49, State: Initialized, Role: FOLLOWER
I20260808 05:02:34.815842 8171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14786700b2f149d5a456718d5825fe49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260808 05:02:34.815951 8171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:34.815995 8171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:34.816051 8171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:34.817091 8171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14786700b2f149d5a456718d5825fe49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260808 05:02:34.817209 8171 leader_election.cc:302] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14786700b2f149d5a456718d5825fe49; no voters:
I20260808 05:02:34.817406 8171 leader_election.cc:288] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:34.817446 8181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:34.817628 8181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [term 1 LEADER]: Becoming Leader. State: Replica: 14786700b2f149d5a456718d5825fe49, State: Running, Role: LEADER
I20260808 05:02:34.817685 8171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:34.817780 8181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14786700b2f149d5a456718d5825fe49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260808 05:02:34.818169 8182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14786700b2f149d5a456718d5825fe49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14786700b2f149d5a456718d5825fe49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } } }
I20260808 05:02:34.818241 8182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:34.818534 8183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14786700b2f149d5a456718d5825fe49. Latest consensus state: current_term: 1 leader_uuid: "14786700b2f149d5a456718d5825fe49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14786700b2f149d5a456718d5825fe49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } } }
I20260808 05:02:34.818861 8183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49 [sys.catalog]: This master's current role is: LEADER
W20260808 05:02:34.819715 8208 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:02:34.819777 8208 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:02:34.819839 8194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:34.820047 8194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:34.821506 8194 catalog_manager.cc:1347] Generated new cluster ID: a5ab66de46044bdea62571f8222cf13a
Aug 08 05:02:34 dist-test-slave-gjw7 krb5kdc[8031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 05:02:34.821570 8194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:34.826354 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:34.814267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:54695 (local address 127.0.0.1:32773)
0808 05:02:34.814464 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:34.814474 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:34.814500 (+ 26us) server_negotiation.cc:407] Connection header received
0808 05:02:34.815420 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:34.815427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:34.815477 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:34.815586 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:34.816452 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:34.817518 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:34.818580 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:34.819128 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:34.819570 (+ 442us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:34.822135 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:34.822139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:34.822140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:34.822152 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:34.824110 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:34.824607 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:34.824610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:34.824610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:34.824751 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:34.825009 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:34.825010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:34.825010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:34.825376 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:34.825929 (+ 553us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:34.826024 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 05:02:34.826135 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260808 05:02:34.827430 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:34.833554 8194 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:34.834105 8194 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:34.836743 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:34.828986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:39351 (local address 127.0.0.1:32773)
0808 05:02:34.829022 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:34.829023 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:34.829094 (+ 71us) server_negotiation.cc:407] Connection header received
0808 05:02:34.829164 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:34.829165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:34.829175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:34.829233 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:34.829881 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:34.830828 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:34.831838 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:34.832206 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:34.832276 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:34.833827 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:34.833829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:34.833829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:34.833837 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:34.835482 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:34.835940 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:34.835941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:34.835941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:34.836003 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:34.836212 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:34.836212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:34.836213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:34.836348 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:34.836441 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:34.836584 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 05:02:34.836647 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":17}
I20260808 05:02:34.841960 8194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14786700b2f149d5a456718d5825fe49: Generated new TSK 0
I20260808 05:02:34.842144 8194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:34.847898 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 8048
W20260808 05:02:34.862017 8029 connection.cc:535] client connection to 127.0.0.1:32773 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32773): Connection reset by peer (error 104)
2026-08-08T05:02:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1457 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:34 dist-test-slave-gjw7 krb5kdc[8226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:34 dist-test-slave-gjw7 krb5kdc[8226](info): set up 2 sockets
Aug 08 05:02:34 dist-test-slave-gjw7 krb5kdc[8226](info): commencing operation
krb5kdc: starting...
W20260808 05:02:36.024654 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:36 dist-test-slave-gjw7 krb5kdc[8226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:36.141463 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39627
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:36.225193 8251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:36.225428 8251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:36.225461 8251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:36.226661 8251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:36.226708 8251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:36.226724 8251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:36.226738 8251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:36.226768 8251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:36.228298 8251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:36.228554 8251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:36.228775 8251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:36.233500 8264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:36.233506 8261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:36.233630 8251 server_base.cc:1034] running on GCE node
W20260808 05:02:36.233706 8262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:36.234009 8251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:36.234287 8251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:36.235447 8251 hybrid_clock.cc:648] HybridClock initialized: now 1786165356235434 us; error 44 us; skew 500 ppm
Aug 08 05:02:36 dist-test-slave-gjw7 krb5kdc[8226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165356, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:36.237882 8251 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:36.238302 8251 webserver.cc:460] Webserver started at http://127.0.0.1:44891/ using document root <none> and password file <none>
I20260808 05:02:36.238519 8251 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:36.238579 8251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:36.238704 8251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:36.239740 8251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "9d7baa317df94f5aa8576e40387ecb75"
format_stamp: "Formatted at 2026-08-08 05:02:36 on dist-test-slave-gjw7"
server_key: "fbfde7ad813479f90d6bb7aee7fa10d0"
server_key_iv: "437a4b98c25a799185d12c19a6db1281"
server_key_version: "encryptionkey@0"
I20260808 05:02:36.240087 8251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "9d7baa317df94f5aa8576e40387ecb75"
format_stamp: "Formatted at 2026-08-08 05:02:36 on dist-test-slave-gjw7"
server_key: "fbfde7ad813479f90d6bb7aee7fa10d0"
server_key_iv: "437a4b98c25a799185d12c19a6db1281"
server_key_version: "encryptionkey@0"
I20260808 05:02:36.241328 8251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 05:02:36.241994 8273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:36.242141 8251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:36.242225 8251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "9d7baa317df94f5aa8576e40387ecb75"
format_stamp: "Formatted at 2026-08-08 05:02:36 on dist-test-slave-gjw7"
server_key: "fbfde7ad813479f90d6bb7aee7fa10d0"
server_key_iv: "437a4b98c25a799185d12c19a6db1281"
server_key_version: "encryptionkey@0"
I20260808 05:02:36.242292 8251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:36.256803 8251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:36.258584 8251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:36.258719 8251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:36.262676 8251 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39627
I20260808 05:02:36.262717 8357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39627 every 8 connection(s)
I20260808 05:02:36.263044 8251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:36.264043 8358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:36.266398 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 8251
I20260808 05:02:36.266528 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:36.266721 3770 external_mini_cluster.cc:1428] Setting key d1d7cd87ab1e53d327419d84cdd03afa
I20260808 05:02:36.267267 8358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75: Bootstrap starting.
I20260808 05:02:36.267998 8358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:36.268334 8358 log.cc:824] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:36.269030 8358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75: No bootstrap required, opened a new log
I20260808 05:02:36.271306 8358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7baa317df94f5aa8576e40387ecb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39627 } }
I20260808 05:02:36.271472 8358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:36.271497 8358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7baa317df94f5aa8576e40387ecb75, State: Initialized, Role: FOLLOWER
I20260808 05:02:36.271615 8358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7baa317df94f5aa8576e40387ecb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39627 } }
I20260808 05:02:36.271699 8358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:36.271741 8358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:36.271795 8358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:36.272516 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:36.267246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:44015 (local address 127.0.0.1:39627)
0808 05:02:36.267440 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:36.267449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:36.267482 (+ 33us) server_negotiation.cc:407] Connection header received
0808 05:02:36.268063 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:36.268074 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:36.268149 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:36.268307 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:36.268509 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:36.269520 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:36.270593 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:36.271009 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:36.271096 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:36.271838 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:36.271843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:36.271843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:36.272043 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:36.272083 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:36.272099 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:36.272186 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:36.272290 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 05:02:36.272321 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260808 05:02:36.272498 8358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7baa317df94f5aa8576e40387ecb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39627 } }
I20260808 05:02:36.272625 8358 leader_election.cc:302] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d7baa317df94f5aa8576e40387ecb75; no voters:
I20260808 05:02:36.272810 8358 leader_election.cc:288] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:36.272861 8375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:36.273003 8375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [term 1 LEADER]: Becoming Leader. State: Replica: 9d7baa317df94f5aa8576e40387ecb75, State: Running, Role: LEADER
I20260808 05:02:36.273103 8358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:36.273095 8375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7baa317df94f5aa8576e40387ecb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39627 } }
I20260808 05:02:36.273524 8377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d7baa317df94f5aa8576e40387ecb75. Latest consensus state: current_term: 1 leader_uuid: "9d7baa317df94f5aa8576e40387ecb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7baa317df94f5aa8576e40387ecb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39627 } } }
I20260808 05:02:36.273578 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:36.273597 8377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:36.273746 8376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d7baa317df94f5aa8576e40387ecb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7baa317df94f5aa8576e40387ecb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39627 } } }
I20260808 05:02:36.273799 8376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:36.274217 8382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:36.274765 8382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:36.276485 8382 catalog_manager.cc:1347] Generated new cluster ID: 5190b1363aba4d0b97fdfc3e071b9f78
I20260808 05:02:36.276537 8382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:36.278997 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:36.275208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:60449 (local address 127.0.0.1:39627)
0808 05:02:36.275243 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:36.275245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:36.275556 (+ 311us) server_negotiation.cc:407] Connection header received
0808 05:02:36.275655 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:36.275656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:36.275666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:36.275693 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:36.275880 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:36.276561 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:36.277591 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:36.277946 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:36.277981 (+ 35us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:36.278719 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:36.278721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:36.278721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:36.278725 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:36.278738 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:36.278742 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:36.278769 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:36.278913 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 05:02:36.278924 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 05:02:36.289522 8382 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:36.290026 8382 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:36.293100 8382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d7baa317df94f5aa8576e40387ecb75: Generated new TSK 0
I20260808 05:02:36.293254 8382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:36.295459 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 8251
W20260808 05:02:36.315822 8221 connection.cc:535] client connection to 127.0.0.1:39627 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39627): Connection reset by peer (error 104)
2026-08-08T05:02:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1453 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:36 dist-test-slave-gjw7 krb5kdc[8413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:36 dist-test-slave-gjw7 krb5kdc[8413](info): set up 2 sockets
Aug 08 05:02:36 dist-test-slave-gjw7 krb5kdc[8413](info): commencing operation
krb5kdc: starting...
W20260808 05:02:37.503733 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:37 dist-test-slave-gjw7 krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:37.625342 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40893
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:37.708339 8438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:37.708541 8438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:37.708573 8438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:37.709733 8438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:37.709777 8438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:37.709805 8438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:37.709823 8438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:37.709847 8438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:37.711351 8438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:37.711587 8438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:37.711786 8438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:37.716332 8445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:37.716467 8448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:37.716332 8451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:37.716638 8438 server_base.cc:1034] running on GCE node
I20260808 05:02:37.716886 8438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:37.717152 8438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:37.718297 8438 hybrid_clock.cc:648] HybridClock initialized: now 1786165357718283 us; error 32 us; skew 500 ppm
Aug 08 05:02:37 dist-test-slave-gjw7 krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165357, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:37.720629 8438 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:37.721088 8438 webserver.cc:460] Webserver started at http://127.0.0.1:43357/ using document root <none> and password file <none>
I20260808 05:02:37.721346 8438 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:37.721400 8438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:37.721527 8438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:37.722548 8438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "5fb65430685e4b80b0f826714e365b4e"
format_stamp: "Formatted at 2026-08-08 05:02:37 on dist-test-slave-gjw7"
server_key: "7db4e47563541ed98032221a9cd9921e"
server_key_iv: "5e1f782be69d402a8ba38146b098fbc6"
server_key_version: "encryptionkey@0"
I20260808 05:02:37.722894 8438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "5fb65430685e4b80b0f826714e365b4e"
format_stamp: "Formatted at 2026-08-08 05:02:37 on dist-test-slave-gjw7"
server_key: "7db4e47563541ed98032221a9cd9921e"
server_key_iv: "5e1f782be69d402a8ba38146b098fbc6"
server_key_version: "encryptionkey@0"
I20260808 05:02:37.724124 8438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:02:37.724802 8464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:37.724953 8438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:37.725031 8438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "5fb65430685e4b80b0f826714e365b4e"
format_stamp: "Formatted at 2026-08-08 05:02:37 on dist-test-slave-gjw7"
server_key: "7db4e47563541ed98032221a9cd9921e"
server_key_iv: "5e1f782be69d402a8ba38146b098fbc6"
server_key_version: "encryptionkey@0"
I20260808 05:02:37.725101 8438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:37.728462 8438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:37.730374 8438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:37.730504 8438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:37.734534 8438 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40893
I20260808 05:02:37.734597 8556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40893 every 8 connection(s)
I20260808 05:02:37.734915 8438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:37.735935 8557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:37.738401 8557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e: Bootstrap starting.
I20260808 05:02:37.738997 8557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:37.739261 8557 log.cc:824] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:37.739933 8557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e: No bootstrap required, opened a new log
I20260808 05:02:37.740831 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 8438
I20260808 05:02:37.740960 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:37.741163 3770 external_mini_cluster.cc:1428] Setting key 579ece5f497e34f3aa180830b6f3b834
I20260808 05:02:37.741990 8557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb65430685e4b80b0f826714e365b4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40893 } }
I20260808 05:02:37.742172 8557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:37.742218 8557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb65430685e4b80b0f826714e365b4e, State: Initialized, Role: FOLLOWER
I20260808 05:02:37.742349 8557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb65430685e4b80b0f826714e365b4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40893 } }
I20260808 05:02:37.742436 8557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:37.742520 8557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:37.742578 8557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:37.743484 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:37.741747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:45853 (local address 127.0.0.1:40893)
0808 05:02:37.741984 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:37.741994 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:37.742029 (+ 35us) server_negotiation.cc:407] Connection header received
0808 05:02:37.742146 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:37.742153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:37.742185 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:37.742383 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:37.743016 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:37.743020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:37.743021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:37.743026 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:37.743074 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:37.743089 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:37.743162 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:37.743211 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 05:02:37.743258 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":127,"threads_started":1}
I20260808 05:02:37.743921 8557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb65430685e4b80b0f826714e365b4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40893 } }
I20260808 05:02:37.744037 8557 leader_election.cc:302] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fb65430685e4b80b0f826714e365b4e; no voters:
I20260808 05:02:37.744199 8557 leader_election.cc:288] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:37.744233 8567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:37.744390 8567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [term 1 LEADER]: Becoming Leader. State: Replica: 5fb65430685e4b80b0f826714e365b4e, State: Running, Role: LEADER
I20260808 05:02:37.744455 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:37.744467 8557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:37.744510 8567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb65430685e4b80b0f826714e365b4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40893 } }
I20260808 05:02:37.744881 8567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fb65430685e4b80b0f826714e365b4e. Latest consensus state: current_term: 1 leader_uuid: "5fb65430685e4b80b0f826714e365b4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb65430685e4b80b0f826714e365b4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40893 } } }
I20260808 05:02:37.744933 8568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fb65430685e4b80b0f826714e365b4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb65430685e4b80b0f826714e365b4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40893 } } }
I20260808 05:02:37.745019 8567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:37.745025 8568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:37.746176 8578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:37.746913 8578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:37.747009 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:37.745979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:60641 (local address 127.0.0.1:40893)
0808 05:02:37.746015 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:37.746017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:37.746134 (+ 117us) server_negotiation.cc:407] Connection header received
0808 05:02:37.746231 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:37.746232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:37.746238 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:37.746283 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:37.746802 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:37.746803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:37.746803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:37.746808 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:37.746824 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:37.746829 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:37.746884 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:37.746942 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 05:02:37.746956 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 05:02:37.748430 8578 catalog_manager.cc:1347] Generated new cluster ID: 0d4775c927ba40f3a88da80e3c87143e
I20260808 05:02:37.748472 8578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:37.753190 8578 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:37.753383 8578 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:37.756009 8578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fb65430685e4b80b0f826714e365b4e: Generated new TSK 0
I20260808 05:02:37.756160 8578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:37.757488 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 8438
2026-08-08T05:02:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:37 dist-test-slave-gjw7 krb5kdc[8607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:37 dist-test-slave-gjw7 krb5kdc[8607](info): set up 2 sockets
Aug 08 05:02:37 dist-test-slave-gjw7 krb5kdc[8607](info): commencing operation
krb5kdc: starting...
W20260808 05:02:38.950210 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:39 dist-test-slave-gjw7 krb5kdc[8607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:39.078830 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39409
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:39.164803 8628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:39.165043 8628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:39.165077 8628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:39.166339 8628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:39.166390 8628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:39.166409 8628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:39.166424 8628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:39.166438 8628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:39.168062 8628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39409
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:39.168358 8628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:39.168601 8628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:39.173655 8648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:39.173700 8637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:39.173863 8628 server_base.cc:1034] running on GCE node
W20260808 05:02:39.173748 8636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:39.174219 8628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:39.174521 8628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:39.175683 8628 hybrid_clock.cc:648] HybridClock initialized: now 1786165359175663 us; error 36 us; skew 500 ppm
Aug 08 05:02:39 dist-test-slave-gjw7 krb5kdc[8607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165359, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:39.178363 8628 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:39.178886 8628 webserver.cc:460] Webserver started at http://127.0.0.1:34241/ using document root <none> and password file <none>
I20260808 05:02:39.179137 8628 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:39.179198 8628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:39.179337 8628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:39.180435 8628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "cd584b7682d44fcf9f71beb026473222"
format_stamp: "Formatted at 2026-08-08 05:02:39 on dist-test-slave-gjw7"
server_key: "83ef1d3f1665ab222a9c98e19dd25b50"
server_key_iv: "dd667a3251158231e2afb97259cf110a"
server_key_version: "encryptionkey@0"
I20260808 05:02:39.180781 8628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "cd584b7682d44fcf9f71beb026473222"
format_stamp: "Formatted at 2026-08-08 05:02:39 on dist-test-slave-gjw7"
server_key: "83ef1d3f1665ab222a9c98e19dd25b50"
server_key_iv: "dd667a3251158231e2afb97259cf110a"
server_key_version: "encryptionkey@0"
I20260808 05:02:39.182294 8628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:02:39.183136 8657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:39.183390 8628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 05:02:39.183488 8628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "cd584b7682d44fcf9f71beb026473222"
format_stamp: "Formatted at 2026-08-08 05:02:39 on dist-test-slave-gjw7"
server_key: "83ef1d3f1665ab222a9c98e19dd25b50"
server_key_iv: "dd667a3251158231e2afb97259cf110a"
server_key_version: "encryptionkey@0"
I20260808 05:02:39.183555 8628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:39.193022 8628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:39.195132 8628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:39.195295 8628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:39.199609 8628 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39409
I20260808 05:02:39.199685 8745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39409 every 8 connection(s)
I20260808 05:02:39.200027 8628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:39.201085 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:39.203651 8746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222: Bootstrap starting.
I20260808 05:02:39.204284 8746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:39.204566 8746 log.cc:824] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:39.204994 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 8628
I20260808 05:02:39.205134 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:39.205349 3770 external_mini_cluster.cc:1428] Setting key a9c537153c4f810800b6b2cbb7f8717a
I20260808 05:02:39.205549 8746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222: No bootstrap required, opened a new log
I20260808 05:02:39.207734 8746 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd584b7682d44fcf9f71beb026473222" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39409 } }
I20260808 05:02:39.207926 8746 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:39.207974 8746 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd584b7682d44fcf9f71beb026473222, State: Initialized, Role: FOLLOWER
I20260808 05:02:39.208112 8746 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd584b7682d44fcf9f71beb026473222" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39409 } }
I20260808 05:02:39.208196 8746 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:39.208228 8746 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:39.208267 8746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:39.209030 8746 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd584b7682d44fcf9f71beb026473222" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39409 } }
I20260808 05:02:39.209144 8746 leader_election.cc:302] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd584b7682d44fcf9f71beb026473222; no voters:
I20260808 05:02:39.209322 8746 leader_election.cc:288] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:39.209357 8757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:39.209501 8757 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [term 1 LEADER]: Becoming Leader. State: Replica: cd584b7682d44fcf9f71beb026473222, State: Running, Role: LEADER
I20260808 05:02:39.209594 8746 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:39.209633 8757 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd584b7682d44fcf9f71beb026473222" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39409 } }
I20260808 05:02:39.210040 8757 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd584b7682d44fcf9f71beb026473222. Latest consensus state: current_term: 1 leader_uuid: "cd584b7682d44fcf9f71beb026473222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd584b7682d44fcf9f71beb026473222" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39409 } } }
I20260808 05:02:39.210173 8757 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:39.210417 8777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:39.210965 8777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:39.211241 8759 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd584b7682d44fcf9f71beb026473222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd584b7682d44fcf9f71beb026473222" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39409 } } }
I20260808 05:02:39.211350 8759 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:39.213835 8777 catalog_manager.cc:1347] Generated new cluster ID: 5e63b5a763724327b9d9bf5d26a32f10
Aug 08 05:02:39 dist-test-slave-gjw7 krb5kdc[8607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 05:02:39.213887 8777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:39.218757 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:39.205914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:47429 (local address 127.0.0.1:39409)
0808 05:02:39.206126 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:39.206137 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:39.206175 (+ 38us) server_negotiation.cc:407] Connection header received
0808 05:02:39.206945 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:39.206956 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:39.207030 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:39.207185 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:39.207798 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:39.209497 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:39.210630 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:39.211238 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:39.211338 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:39.214448 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:39.214453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:39.214454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:39.214467 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:39.216466 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:39.216919 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:39.216922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:39.216923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:39.217038 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:39.217247 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:39.217248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:39.217248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:39.217613 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:39.218315 (+ 702us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:39.218413 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 05:02:39.218524 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260808 05:02:39.219863 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:39.221397 8777 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:39.221966 8777 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:39.228518 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:39.221180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:51121 (local address 127.0.0.1:39409)
0808 05:02:39.221370 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:39.221371 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:39.221549 (+ 178us) server_negotiation.cc:407] Connection header received
0808 05:02:39.221612 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:39.221613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:39.221624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:39.221680 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:39.222336 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:39.223074 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:39.224080 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:39.224394 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:39.224444 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:39.225849 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:39.225851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:39.225851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:39.225858 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:39.227369 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:39.227781 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:39.227782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:39.227782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:39.227836 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:39.228033 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:39.228034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:39.228034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:39.228147 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:39.228244 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:39.228362 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 05:02:39.228423 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260808 05:02:39.231945 8777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd584b7682d44fcf9f71beb026473222: Generated new TSK 0
I20260808 05:02:39.232136 8777 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:39.236665 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 8628
W20260808 05:02:39.251065 8604 connection.cc:535] client connection to 127.0.0.1:39409 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39409): Connection reset by peer (error 104)
2026-08-08T05:02:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:39 dist-test-slave-gjw7 krb5kdc[8812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:39 dist-test-slave-gjw7 krb5kdc[8812](info): set up 2 sockets
Aug 08 05:02:39 dist-test-slave-gjw7 krb5kdc[8812](info): commencing operation
krb5kdc: starting...
W20260808 05:02:40.429152 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:40 dist-test-slave-gjw7 krb5kdc[8812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:40.547983 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:40.630324 8829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:40.630530 8829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:40.630561 8829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:40.631693 8829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:40.631737 8829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:40.631757 8829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:40.631773 8829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:40.631788 8829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:40.633230 8829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:40.633500 8829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:40.633710 8829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:40.638437 8837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:40.638554 8829 server_base.cc:1034] running on GCE node
W20260808 05:02:40.638718 8838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:40.638460 8843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:40.639068 8829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:40.639346 8829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:40.640491 8829 hybrid_clock.cc:648] HybridClock initialized: now 1786165360640473 us; error 35 us; skew 500 ppm
Aug 08 05:02:40 dist-test-slave-gjw7 krb5kdc[8812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165360, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:40.642906 8829 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:40.643324 8829 webserver.cc:460] Webserver started at http://127.0.0.1:42929/ using document root <none> and password file <none>
I20260808 05:02:40.643546 8829 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:40.643594 8829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:40.643718 8829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:40.644762 8829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "d31501e22689415cb78823354b5703f5"
format_stamp: "Formatted at 2026-08-08 05:02:40 on dist-test-slave-gjw7"
server_key: "db4086942c3cef647727723558c82d1b"
server_key_iv: "6303bd03359e7556094ee7978bc5053b"
server_key_version: "encryptionkey@0"
I20260808 05:02:40.645097 8829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "d31501e22689415cb78823354b5703f5"
format_stamp: "Formatted at 2026-08-08 05:02:40 on dist-test-slave-gjw7"
server_key: "db4086942c3cef647727723558c82d1b"
server_key_iv: "6303bd03359e7556094ee7978bc5053b"
server_key_version: "encryptionkey@0"
I20260808 05:02:40.646466 8829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:40.647143 8853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:40.647295 8829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:40.647374 8829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "d31501e22689415cb78823354b5703f5"
format_stamp: "Formatted at 2026-08-08 05:02:40 on dist-test-slave-gjw7"
server_key: "db4086942c3cef647727723558c82d1b"
server_key_iv: "6303bd03359e7556094ee7978bc5053b"
server_key_version: "encryptionkey@0"
I20260808 05:02:40.647439 8829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:40.672814 8829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:40.674613 8829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:40.674763 8829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:40.678814 8829 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46823
I20260808 05:02:40.678860 8939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46823 every 8 connection(s)
I20260808 05:02:40.679217 8829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:40.680107 8940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:40.682559 8940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5: Bootstrap starting.
I20260808 05:02:40.682832 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 8829
I20260808 05:02:40.682958 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:40.683133 3770 external_mini_cluster.cc:1428] Setting key f16aacbe0616c54e5d0d581f72e20731
I20260808 05:02:40.683230 8940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:40.683588 8940 log.cc:824] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:40.684423 8940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5: No bootstrap required, opened a new log
I20260808 05:02:40.686857 8940 raft_consensus.cc:348] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31501e22689415cb78823354b5703f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } }
I20260808 05:02:40.687011 8940 raft_consensus.cc:374] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:40.687044 8940 raft_consensus.cc:729] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31501e22689415cb78823354b5703f5, State: Initialized, Role: FOLLOWER
I20260808 05:02:40.687177 8940 consensus_queue.cc:260] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31501e22689415cb78823354b5703f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } }
I20260808 05:02:40.687253 8940 raft_consensus.cc:388] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:40.687290 8940 raft_consensus.cc:482] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:40.687342 8940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:40.688074 8940 raft_consensus.cc:504] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31501e22689415cb78823354b5703f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } }
I20260808 05:02:40.688197 8940 leader_election.cc:302] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d31501e22689415cb78823354b5703f5; no voters:
I20260808 05:02:40.688385 8940 leader_election.cc:288] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:40.688413 8954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:40.688544 8954 raft_consensus.cc:686] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [term 1 LEADER]: Becoming Leader. State: Replica: d31501e22689415cb78823354b5703f5, State: Running, Role: LEADER
I20260808 05:02:40.688663 8940 sys_catalog.cc:564] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:40.688678 8954 consensus_queue.cc:237] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31501e22689415cb78823354b5703f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } }
I20260808 05:02:40.689031 8954 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d31501e22689415cb78823354b5703f5. Latest consensus state: current_term: 1 leader_uuid: "d31501e22689415cb78823354b5703f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31501e22689415cb78823354b5703f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } } }
I20260808 05:02:40.689109 8954 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:40.689318 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:40.683665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:50201 (local address 127.0.0.1:46823)
0808 05:02:40.683897 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:40.683907 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:40.683943 (+ 36us) server_negotiation.cc:407] Connection header received
0808 05:02:40.684505 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:40.684515 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:40.684591 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:40.684739 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:40.684907 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:40.686272 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:40.687022 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:40.687618 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:40.687727 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:40.688399 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:40.688405 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:40.688405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:40.688630 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:40.688670 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:40.688686 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:40.688774 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:40.688993 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 05:02:40.689023 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":101,"threads_started":1}
I20260808 05:02:40.689260 8955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d31501e22689415cb78823354b5703f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31501e22689415cb78823354b5703f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } } }
I20260808 05:02:40.689505 8955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:40.689795 8961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:40.690343 8961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:40.690368 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:40.692329 8961 catalog_manager.cc:1347] Generated new cluster ID: d4636b298e5d4ae986d96b8713da84f5
I20260808 05:02:40.692379 8961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:40.695349 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:40.691963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:39179 (local address 127.0.0.1:46823)
0808 05:02:40.691998 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:40.692000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:40.692106 (+ 106us) server_negotiation.cc:407] Connection header received
0808 05:02:40.692147 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:40.692148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:40.692158 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:40.692201 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:40.692370 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:40.693089 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:40.694100 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:40.694409 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:40.694485 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:40.695126 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:40.695128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:40.695128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:40.695133 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:40.695148 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:40.695152 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:40.695215 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:40.695268 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 05:02:40.695278 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 05:02:40.708365 8961 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:40.708911 8961 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:40.715054 8961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d31501e22689415cb78823354b5703f5: Generated new TSK 0
I20260808 05:02:40.715226 8961 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:40.718735 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 8829
W20260808 05:02:40.752903 8808 connection.cc:535] client connection to 127.0.0.1:46823 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46823): Connection reset by peer (error 104)
2026-08-08T05:02:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:40 dist-test-slave-gjw7 krb5kdc[8998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:40 dist-test-slave-gjw7 krb5kdc[8998](info): set up 2 sockets
Aug 08 05:02:40 dist-test-slave-gjw7 krb5kdc[8998](info): commencing operation
krb5kdc: starting...
W20260808 05:02:41.921448 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:41 dist-test-slave-gjw7 krb5kdc[8998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 05:02:42.037781 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:42.119519 9026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:42.119717 9026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:42.119737 9026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:42.120899 9026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:42.120942 9026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:42.120955 9026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:42.120966 9026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:42.120976 9026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:42.122411 9026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38905
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:42.122602 9026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:42.122771 9026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:42.127398 9033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:42.127384 9034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:42.127398 9037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:42.127707 9026 server_base.cc:1034] running on GCE node
I20260808 05:02:42.127950 9026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:42.128211 9026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:42.129351 9026 hybrid_clock.cc:648] HybridClock initialized: now 1786165362129334 us; error 27 us; skew 500 ppm
Aug 08 05:02:42 dist-test-slave-gjw7 krb5kdc[8998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165362, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:42.134141 9026 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 05:02:42.134569 9026 webserver.cc:460] Webserver started at http://127.0.0.1:35495/ using document root <none> and password file <none>
I20260808 05:02:42.134768 9026 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:42.134815 9026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:42.134948 9026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:42.136013 9026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "ea79c93d333146c38055421b78402907"
format_stamp: "Formatted at 2026-08-08 05:02:42 on dist-test-slave-gjw7"
server_key: "b679d4d6ee65bbea2ed015430b68ed24"
server_key_iv: "26a72c4a88e9263e9a197feb189fd84f"
server_key_version: "encryptionkey@0"
I20260808 05:02:42.136348 9026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "ea79c93d333146c38055421b78402907"
format_stamp: "Formatted at 2026-08-08 05:02:42 on dist-test-slave-gjw7"
server_key: "b679d4d6ee65bbea2ed015430b68ed24"
server_key_iv: "26a72c4a88e9263e9a197feb189fd84f"
server_key_version: "encryptionkey@0"
I20260808 05:02:42.137606 9026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:02:42.138262 9053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:42.138391 9026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:42.138478 9026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "ea79c93d333146c38055421b78402907"
format_stamp: "Formatted at 2026-08-08 05:02:42 on dist-test-slave-gjw7"
server_key: "b679d4d6ee65bbea2ed015430b68ed24"
server_key_iv: "26a72c4a88e9263e9a197feb189fd84f"
server_key_version: "encryptionkey@0"
I20260808 05:02:42.138535 9026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:42.141765 9026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:42.143435 9026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:42.143564 9026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:42.147401 9026 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38905
I20260808 05:02:42.147430 9141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38905 every 8 connection(s)
I20260808 05:02:42.147720 9026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:42.148538 9143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:42.150817 9143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907: Bootstrap starting.
I20260808 05:02:42.151371 9143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:42.151602 9143 log.cc:824] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:42.151925 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9026
I20260808 05:02:42.152035 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:42.152230 3770 external_mini_cluster.cc:1428] Setting key 9c53fefcc44f91c004fa3f692142c70e
I20260808 05:02:42.152307 9143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907: No bootstrap required, opened a new log
I20260808 05:02:42.154484 9143 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea79c93d333146c38055421b78402907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38905 } }
I20260808 05:02:42.154650 9143 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:42.154701 9143 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea79c93d333146c38055421b78402907, State: Initialized, Role: FOLLOWER
I20260808 05:02:42.154825 9143 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea79c93d333146c38055421b78402907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38905 } }
I20260808 05:02:42.154919 9143 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:42.154958 9143 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:42.154982 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:42.152693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:42155 (local address 127.0.0.1:38905)
0808 05:02:42.152905 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:42.152916 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:42.152953 (+ 37us) server_negotiation.cc:407] Connection header received
0808 05:02:42.153613 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:42.153622 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:42.153659 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:42.153801 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:42.154336 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:42.154340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:42.154340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:42.154344 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:42.154388 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:42.154402 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:42.154473 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:42.154531 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 05:02:42.154573 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260808 05:02:42.155004 9143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:42.155665 9143 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea79c93d333146c38055421b78402907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38905 } }
I20260808 05:02:42.155771 9143 leader_election.cc:302] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea79c93d333146c38055421b78402907; no voters:
I20260808 05:02:42.155864 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:42.155937 9143 leader_election.cc:288] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:42.155970 9148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:42.156077 9148 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [term 1 LEADER]: Becoming Leader. State: Replica: ea79c93d333146c38055421b78402907, State: Running, Role: LEADER
I20260808 05:02:42.156200 9143 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:42.156219 9148 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea79c93d333146c38055421b78402907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38905 } }
I20260808 05:02:42.156594 9148 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea79c93d333146c38055421b78402907. Latest consensus state: current_term: 1 leader_uuid: "ea79c93d333146c38055421b78402907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea79c93d333146c38055421b78402907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38905 } } }
I20260808 05:02:42.156646 9151 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea79c93d333146c38055421b78402907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea79c93d333146c38055421b78402907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38905 } } }
I20260808 05:02:42.156731 9148 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:42.156744 9151 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:42.156987 9162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:42.157560 9162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:42.158953 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:42.157389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:40435 (local address 127.0.0.1:38905)
0808 05:02:42.157420 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:42.157421 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:42.157840 (+ 419us) server_negotiation.cc:407] Connection header received
0808 05:02:42.157937 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:42.157938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:42.157942 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:42.157985 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:42.158763 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:42.158764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:42.158765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:42.158769 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:42.158783 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:42.158786 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:42.158832 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:42.158882 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 05:02:42.158894 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 05:02:42.159202 9162 catalog_manager.cc:1347] Generated new cluster ID: 13b637ce6cf04dc48d2d85c364930030
I20260808 05:02:42.159248 9162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:42.166498 9162 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:42.166643 9162 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:42.187109 9162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea79c93d333146c38055421b78402907: Generated new TSK 0
I20260808 05:02:42.187309 9162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:42.194612 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9026
2026-08-08T05:02:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:42 dist-test-slave-gjw7 krb5kdc[9191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:42 dist-test-slave-gjw7 krb5kdc[9191](info): set up 2 sockets
Aug 08 05:02:42 dist-test-slave-gjw7 krb5kdc[9191](info): commencing operation
krb5kdc: starting...
W20260808 05:02:43.369022 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:43 dist-test-slave-gjw7 krb5kdc[9191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:43.485561 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38627
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:38627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:43.569024 9214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:43.569242 9214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:43.569273 9214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:43.570467 9214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:43.570513 9214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:43.570533 9214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:43.570549 9214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:43.570564 9214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:43.572058 9214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:38627
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:38627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:43.572319 9214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:43.572530 9214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:43.577154 9227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:43.577153 9222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:43.577227 9225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:43.577358 9214 server_base.cc:1034] running on GCE node
I20260808 05:02:43.577673 9214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:43.577942 9214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:43.579072 9214 hybrid_clock.cc:648] HybridClock initialized: now 1786165363579058 us; error 31 us; skew 500 ppm
Aug 08 05:02:43 dist-test-slave-gjw7 krb5kdc[9191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165363, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:43.581554 9214 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:43.581956 9214 webserver.cc:460] Webserver started at http://127.3.174.190:46353/ using document root <none> and password file <none>
I20260808 05:02:43.582160 9214 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:43.582204 9214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:43.582310 9214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:43.583348 9214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "ff3d810bda9643b29587c999b4865823"
format_stamp: "Formatted at 2026-08-08 05:02:43 on dist-test-slave-gjw7"
server_key: "8a6f1bb293ff1977852d90738ca41761"
server_key_iv: "2096c7f35524139fd367a36cb38cabef"
server_key_version: "encryptionkey@0"
I20260808 05:02:43.583683 9214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "ff3d810bda9643b29587c999b4865823"
format_stamp: "Formatted at 2026-08-08 05:02:43 on dist-test-slave-gjw7"
server_key: "8a6f1bb293ff1977852d90738ca41761"
server_key_iv: "2096c7f35524139fd367a36cb38cabef"
server_key_version: "encryptionkey@0"
I20260808 05:02:43.584918 9214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:43.585619 9243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:43.585768 9214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:43.585856 9214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "ff3d810bda9643b29587c999b4865823"
format_stamp: "Formatted at 2026-08-08 05:02:43 on dist-test-slave-gjw7"
server_key: "8a6f1bb293ff1977852d90738ca41761"
server_key_iv: "2096c7f35524139fd367a36cb38cabef"
server_key_version: "encryptionkey@0"
I20260808 05:02:43.585919 9214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:43.608577 9214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:43.610741 9214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:43.610880 9214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:43.614727 9214 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:38627
I20260808 05:02:43.614770 9337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:38627 every 8 connection(s)
I20260808 05:02:43.615108 9214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:43.616014 9341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:43.618399 9341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823: Bootstrap starting.
I20260808 05:02:43.618995 9341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:43.619269 9341 log.cc:824] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:43.619971 9341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823: No bootstrap required, opened a new log
I20260808 05:02:43.620126 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9214
I20260808 05:02:43.620234 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:43.620409 3770 external_mini_cluster.cc:1428] Setting key a0453198b9d5335daf07ba59a68e3d4b
I20260808 05:02:43.622129 9341 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3d810bda9643b29587c999b4865823" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38627 } }
I20260808 05:02:43.622350 9341 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:43.622401 9341 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff3d810bda9643b29587c999b4865823, State: Initialized, Role: FOLLOWER
I20260808 05:02:43.622535 9341 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3d810bda9643b29587c999b4865823" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38627 } }
I20260808 05:02:43.622627 9341 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:43.622668 9341 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:43.622716 9341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:43.623239 9341 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3d810bda9643b29587c999b4865823" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38627 } }
I20260808 05:02:43.623363 9341 leader_election.cc:302] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff3d810bda9643b29587c999b4865823; no voters:
I20260808 05:02:43.623520 9341 leader_election.cc:288] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:43.623543 9348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:43.623651 9348 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [term 1 LEADER]: Becoming Leader. State: Replica: ff3d810bda9643b29587c999b4865823, State: Running, Role: LEADER
I20260808 05:02:43.623783 9341 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:43.623798 9348 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3d810bda9643b29587c999b4865823" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38627 } }
I20260808 05:02:43.624141 9348 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff3d810bda9643b29587c999b4865823. Latest consensus state: current_term: 1 leader_uuid: "ff3d810bda9643b29587c999b4865823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3d810bda9643b29587c999b4865823" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38627 } } }
I20260808 05:02:43.624370 9348 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:43.624297 9350 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff3d810bda9643b29587c999b4865823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3d810bda9643b29587c999b4865823" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 38627 } } }
I20260808 05:02:43.624774 9350 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:43.624948 9362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:43.625512 9362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:43.627254 9362 catalog_manager.cc:1347] Generated new cluster ID: a1a2a42104b74ac8a7636e47c949049d
I20260808 05:02:43.627313 9362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 05:02:43 dist-test-slave-gjw7 krb5kdc[9191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:43.631654 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:43.620889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39595 (local address 127.3.174.190:38627)
0808 05:02:43.621074 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:43.621084 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:43.621110 (+ 26us) server_negotiation.cc:407] Connection header received
0808 05:02:43.621694 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:43.621701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:43.621752 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:43.621871 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:43.622640 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:43.624170 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:43.625245 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:43.625936 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:43.626043 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:43.628513 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:43.628517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:43.628517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:43.628531 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:43.630086 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:43.630447 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:43.630450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:43.630451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:43.630529 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:43.630734 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:43.630735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:43.630735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:43.630957 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:43.631102 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:43.631246 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 05:02:43.631356 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260808 05:02:43.632571 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:43.645200 9362 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:43.645201 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:43.639561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50391 (local address 127.3.174.190:38627)
0808 05:02:43.639597 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:43.639599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:43.639719 (+ 120us) server_negotiation.cc:407] Connection header received
0808 05:02:43.639782 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:43.639783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:43.639793 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:43.639837 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:43.640329 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:43.640820 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:43.641547 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:43.641738 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:43.641792 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:43.642848 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:43.642849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:43.642850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:43.642857 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:43.644043 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:43.644372 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:43.644373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:43.644374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:43.644422 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:43.644607 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:43.644608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:43.644608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:43.644717 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:43.644808 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:43.645053 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 05:02:43.645116 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 05:02:43.645965 9362 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:43.648543 9362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff3d810bda9643b29587c999b4865823: Generated new TSK 0
I20260808 05:02:43.648711 9362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:43.654981 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9214
W20260808 05:02:43.669449 9188 connection.cc:535] client connection to 127.3.174.190:38627 recv error: Network error: failed to read from TLS socket (remote: 127.3.174.190:38627): Connection reset by peer (error 104)
2026-08-08T05:02:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:43 dist-test-slave-gjw7 krb5kdc[9389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:43 dist-test-slave-gjw7 krb5kdc[9389](info): set up 2 sockets
Aug 08 05:02:43 dist-test-slave-gjw7 krb5kdc[9389](info): commencing operation
krb5kdc: starting...
W20260808 05:02:44.877595 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:44 dist-test-slave-gjw7 krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:45.047135 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35989
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:35989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:45.130394 9410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:45.130584 9410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:45.130604 9410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:45.131723 9410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:45.131760 9410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:45.131773 9410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:45.131784 9410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:45.131800 9410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:45.133195 9410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:35989
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:35989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:45.133417 9410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:45.133600 9410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:45.138161 9421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:45.138167 9422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:45.138217 9410 server_base.cc:1034] running on GCE node
W20260808 05:02:45.138197 9424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:45.138579 9410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:45.138842 9410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:45.139981 9410 hybrid_clock.cc:648] HybridClock initialized: now 1786165365139965 us; error 28 us; skew 500 ppm
Aug 08 05:02:45 dist-test-slave-gjw7 krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165365, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:45.142424 9410 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:45.142835 9410 webserver.cc:460] Webserver started at http://127.3.174.190:40747/ using document root <none> and password file <none>
I20260808 05:02:45.143038 9410 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:45.143076 9410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:45.143179 9410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:45.144164 9410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "39866acec14c48a3be885797171734ca"
format_stamp: "Formatted at 2026-08-08 05:02:45 on dist-test-slave-gjw7"
server_key: "3dd990901837c27119c26d77bb6042d9"
server_key_iv: "3503f1ab813cafde3081b71b0931d5b9"
server_key_version: "encryptionkey@0"
I20260808 05:02:45.144444 9410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "39866acec14c48a3be885797171734ca"
format_stamp: "Formatted at 2026-08-08 05:02:45 on dist-test-slave-gjw7"
server_key: "3dd990901837c27119c26d77bb6042d9"
server_key_iv: "3503f1ab813cafde3081b71b0931d5b9"
server_key_version: "encryptionkey@0"
I20260808 05:02:45.145644 9410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:02:45.146311 9439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:45.146468 9410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:45.146565 9410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "39866acec14c48a3be885797171734ca"
format_stamp: "Formatted at 2026-08-08 05:02:45 on dist-test-slave-gjw7"
server_key: "3dd990901837c27119c26d77bb6042d9"
server_key_iv: "3503f1ab813cafde3081b71b0931d5b9"
server_key_version: "encryptionkey@0"
I20260808 05:02:45.146625 9410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:45.156522 9410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:45.158239 9410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:45.158345 9410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:45.162135 9410 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:35989
I20260808 05:02:45.162196 9530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:35989 every 8 connection(s)
I20260808 05:02:45.162497 9410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:45.163343 9532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:45.165712 9532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca: Bootstrap starting.
I20260808 05:02:45.166277 9532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:45.166536 9532 log.cc:824] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:45.167241 9532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca: No bootstrap required, opened a new log
I20260808 05:02:45.168859 9532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39866acec14c48a3be885797171734ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35989 } }
I20260808 05:02:45.169000 9532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:45.169034 9532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39866acec14c48a3be885797171734ca, State: Initialized, Role: FOLLOWER
I20260808 05:02:45.169180 9532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39866acec14c48a3be885797171734ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35989 } }
I20260808 05:02:45.169251 9532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:45.169283 9532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:45.169363 9532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:45.170177 9532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39866acec14c48a3be885797171734ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35989 } }
I20260808 05:02:45.170279 9532 leader_election.cc:302] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39866acec14c48a3be885797171734ca; no voters:
I20260808 05:02:45.170444 9532 leader_election.cc:288] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:45.170485 9537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:45.170640 9537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [term 1 LEADER]: Becoming Leader. State: Replica: 39866acec14c48a3be885797171734ca, State: Running, Role: LEADER
I20260808 05:02:45.170714 9532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:45.170750 9537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39866acec14c48a3be885797171734ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35989 } }
I20260808 05:02:45.171118 9544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39866acec14c48a3be885797171734ca. Latest consensus state: current_term: 1 leader_uuid: "39866acec14c48a3be885797171734ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39866acec14c48a3be885797171734ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35989 } } }
I20260808 05:02:45.171211 9544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:45.171108 9543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39866acec14c48a3be885797171734ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39866acec14c48a3be885797171734ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 35989 } } }
I20260808 05:02:45.171345 9543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:45.171434 9554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:45.171890 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9410
I20260808 05:02:45.172017 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:45.172192 9554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:45.172223 3770 external_mini_cluster.cc:1428] Setting key 17f3baba321de85b33e8475d914a68f3
I20260808 05:02:45.173992 9554 catalog_manager.cc:1347] Generated new cluster ID: 724b0e08491240e9835931c4bdfcd74f
I20260808 05:02:45.174046 9554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:45.178261 9566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:45.172695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43799 (local address 127.3.174.190:35989)
0808 05:02:45.173086 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:45.173094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:45.173118 (+ 24us) server_negotiation.cc:407] Connection header received
0808 05:02:45.173212 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:45.173217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:45.173270 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:45.173431 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:45.173692 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:45.175090 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:45.176186 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:45.176778 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:45.176871 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:45.177590 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:45.177595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:45.177595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:45.177802 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:45.177840 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:45.177856 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:45.177929 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:45.178031 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 05:02:45.178057 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":75,"threads_started":1}
I20260808 05:02:45.179116 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:45.183810 9566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:45.180502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35331 (local address 127.3.174.190:35989)
0808 05:02:45.180531 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:45.180533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:45.180634 (+ 101us) server_negotiation.cc:407] Connection header received
0808 05:02:45.180713 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:45.180714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:45.180722 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:45.180754 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:45.180920 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:45.181638 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:45.182586 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:45.182886 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:45.182919 (+ 33us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:45.183573 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:45.183574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:45.183575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:45.183578 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:45.183590 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:45.183595 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:45.183620 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:45.183720 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 05:02:45.183742 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 05:02:45.186669 9554 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:45.187129 9554 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:45.199313 9554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39866acec14c48a3be885797171734ca: Generated new TSK 0
I20260808 05:02:45.199491 9554 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:45.202217 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9410
W20260808 05:02:45.217808 9386 connection.cc:535] client connection to 127.3.174.190:35989 recv error: Network error: failed to read from TLS socket (remote: 127.3.174.190:35989): Connection reset by peer (error 104)
2026-08-08T05:02:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:45 dist-test-slave-gjw7 krb5kdc[9586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:45 dist-test-slave-gjw7 krb5kdc[9586](info): set up 2 sockets
Aug 08 05:02:45 dist-test-slave-gjw7 krb5kdc[9586](info): commencing operation
krb5kdc: starting...
W20260808 05:02:46.388599 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:46 dist-test-slave-gjw7 krb5kdc[9586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:46.513881 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:41239
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:41239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:46.599090 9606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:46.599318 9606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:46.599352 9606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:46.600514 9606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:46.600566 9606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:46.600590 9606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:46.600608 9606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:46.600627 9606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:46.602187 9606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:41239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:41239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:46.602437 9606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:46.602666 9606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:46.607474 9613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:46.607550 9615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:46.607514 9617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:46.607746 9606 server_base.cc:1034] running on GCE node
I20260808 05:02:46.607999 9606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:46.608304 9606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:46.609448 9606 hybrid_clock.cc:648] HybridClock initialized: now 1786165366609421 us; error 44 us; skew 500 ppm
Aug 08 05:02:46 dist-test-slave-gjw7 krb5kdc[9586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165366, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:46.612028 9606 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:46.612453 9606 webserver.cc:460] Webserver started at http://127.3.174.190:37731/ using document root <none> and password file <none>
I20260808 05:02:46.612686 9606 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:46.612735 9606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:46.612869 9606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:46.613993 9606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "9260bc7afc4c44b6992d942d4814ef83"
format_stamp: "Formatted at 2026-08-08 05:02:46 on dist-test-slave-gjw7"
server_key: "8d4a6f11984d3ed36e365c36e832f174"
server_key_iv: "01f775ecccf48a9c0108ff8dee7330b5"
server_key_version: "encryptionkey@0"
I20260808 05:02:46.614336 9606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "9260bc7afc4c44b6992d942d4814ef83"
format_stamp: "Formatted at 2026-08-08 05:02:46 on dist-test-slave-gjw7"
server_key: "8d4a6f11984d3ed36e365c36e832f174"
server_key_iv: "01f775ecccf48a9c0108ff8dee7330b5"
server_key_version: "encryptionkey@0"
I20260808 05:02:46.615604 9606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 05:02:46.616420 9637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:46.616598 9606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:46.616688 9606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "9260bc7afc4c44b6992d942d4814ef83"
format_stamp: "Formatted at 2026-08-08 05:02:46 on dist-test-slave-gjw7"
server_key: "8d4a6f11984d3ed36e365c36e832f174"
server_key_iv: "01f775ecccf48a9c0108ff8dee7330b5"
server_key_version: "encryptionkey@0"
I20260808 05:02:46.616750 9606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:46.620069 9606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:46.622231 9606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:46.622367 9606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:46.626268 9606 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:41239
I20260808 05:02:46.626308 9722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:41239 every 8 connection(s)
I20260808 05:02:46.626617 9606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:46.627590 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:46.628916 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9606
I20260808 05:02:46.629033 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:46.629212 3770 external_mini_cluster.cc:1428] Setting key a760453bb26714f9441c761cc218db5e
I20260808 05:02:46.630224 9723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83: Bootstrap starting.
I20260808 05:02:46.630885 9723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:46.631253 9723 log.cc:824] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:46.631734 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:46.629755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45385 (local address 127.3.174.190:41239)
0808 05:02:46.629955 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:46.629964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:46.629989 (+ 25us) server_negotiation.cc:407] Connection header received
0808 05:02:46.630430 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:46.630438 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:46.630468 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:46.630593 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:46.631292 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:46.631296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:46.631297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:46.631301 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:46.631345 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:46.631357 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:46.631442 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:46.631500 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 05:02:46.631544 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260808 05:02:46.632050 9723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83: No bootstrap required, opened a new log
I20260808 05:02:46.632673 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:46.634495 9723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260bc7afc4c44b6992d942d4814ef83" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41239 } }
I20260808 05:02:46.634698 9723 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:46.634735 9723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9260bc7afc4c44b6992d942d4814ef83, State: Initialized, Role: FOLLOWER
I20260808 05:02:46.634876 9723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260bc7afc4c44b6992d942d4814ef83" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41239 } }
I20260808 05:02:46.634989 9723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:46.635061 9723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:46.635116 9723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:46.635283 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:46.634295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36739 (local address 127.3.174.190:41239)
0808 05:02:46.634354 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:46.634357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:46.634377 (+ 20us) server_negotiation.cc:407] Connection header received
0808 05:02:46.634437 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:46.634438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:46.634441 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:46.634476 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:46.635039 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:46.635041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:46.635041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:46.635046 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:46.635061 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:46.635066 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:46.635119 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:46.635209 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 05:02:46.635221 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 05:02:46.635761 9723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260bc7afc4c44b6992d942d4814ef83" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41239 } }
I20260808 05:02:46.635862 9723 leader_election.cc:302] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9260bc7afc4c44b6992d942d4814ef83; no voters:
I20260808 05:02:46.636042 9723 leader_election.cc:288] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:46.636070 9740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:46.636202 9740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [term 1 LEADER]: Becoming Leader. State: Replica: 9260bc7afc4c44b6992d942d4814ef83, State: Running, Role: LEADER
I20260808 05:02:46.636291 9723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:46.636348 9740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260bc7afc4c44b6992d942d4814ef83" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41239 } }
I20260808 05:02:46.636719 9742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9260bc7afc4c44b6992d942d4814ef83. Latest consensus state: current_term: 1 leader_uuid: "9260bc7afc4c44b6992d942d4814ef83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260bc7afc4c44b6992d942d4814ef83" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41239 } } }
I20260808 05:02:46.636713 9741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9260bc7afc4c44b6992d942d4814ef83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260bc7afc4c44b6992d942d4814ef83" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41239 } } }
I20260808 05:02:46.636822 9742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:46.636835 9741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83 [sys.catalog]: This master's current role is: LEADER
W20260808 05:02:46.637987 9771 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:02:46.638033 9771 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:02:46.638087 9756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:46.638254 9756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:46.639657 9756 catalog_manager.cc:1347] Generated new cluster ID: ca3eab0af41248bebaee50f9c8874064
I20260808 05:02:46.639707 9756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:46.644006 9756 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:46.644169 9756 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:46.657922 9756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9260bc7afc4c44b6992d942d4814ef83: Generated new TSK 0
I20260808 05:02:46.658138 9756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:46.661608 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9606
2026-08-08T05:02:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:46 dist-test-slave-gjw7 krb5kdc[9781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:46 dist-test-slave-gjw7 krb5kdc[9781](info): set up 2 sockets
Aug 08 05:02:46 dist-test-slave-gjw7 krb5kdc[9781](info): commencing operation
krb5kdc: starting...
W20260808 05:02:47.872004 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:47 dist-test-slave-gjw7 krb5kdc[9781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:47.996420 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:41595
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:41595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:48.082422 9811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:48.082647 9811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:48.082680 9811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:48.083861 9811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:48.083907 9811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:48.083926 9811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:48.083942 9811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:48.083958 9811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:48.085574 9811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:41595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:41595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:48.085841 9811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:48.086066 9811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:48.091028 9820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:48.091060 9822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:48.091084 9811 server_base.cc:1034] running on GCE node
W20260808 05:02:48.091028 9819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:48.091569 9811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:48.091866 9811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:48.093009 9811 hybrid_clock.cc:648] HybridClock initialized: now 1786165368092964 us; error 62 us; skew 500 ppm
Aug 08 05:02:48 dist-test-slave-gjw7 krb5kdc[9781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165368, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:48.099467 9811 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:48.099987 9811 webserver.cc:460] Webserver started at http://127.3.174.190:46315/ using document root <none> and password file <none>
I20260808 05:02:48.100226 9811 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:48.100286 9811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:48.100420 9811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:48.101575 9811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "8f5178a81d1e459e941846cbae8797ff"
format_stamp: "Formatted at 2026-08-08 05:02:48 on dist-test-slave-gjw7"
server_key: "031e15b997e11840931f7523f2a8f07d"
server_key_iv: "82275e2c09d24f8968749a778af5aea5"
server_key_version: "encryptionkey@0"
I20260808 05:02:48.101951 9811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "8f5178a81d1e459e941846cbae8797ff"
format_stamp: "Formatted at 2026-08-08 05:02:48 on dist-test-slave-gjw7"
server_key: "031e15b997e11840931f7523f2a8f07d"
server_key_iv: "82275e2c09d24f8968749a778af5aea5"
server_key_version: "encryptionkey@0"
I20260808 05:02:48.103425 9811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 05:02:48.104344 9835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:48.104555 9811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:48.104651 9811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "8f5178a81d1e459e941846cbae8797ff"
format_stamp: "Formatted at 2026-08-08 05:02:48 on dist-test-slave-gjw7"
server_key: "031e15b997e11840931f7523f2a8f07d"
server_key_iv: "82275e2c09d24f8968749a778af5aea5"
server_key_version: "encryptionkey@0"
I20260808 05:02:48.104717 9811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:48.119336 9811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:48.121512 9811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:48.121661 9811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:48.125756 9811 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:41595
I20260808 05:02:48.125797 9931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:41595 every 8 connection(s)
I20260808 05:02:48.126225 9811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:48.127156 9933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:48.129606 9933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff: Bootstrap starting.
I20260808 05:02:48.130196 9933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:48.130456 9933 log.cc:824] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:48.131134 9933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff: No bootstrap required, opened a new log
I20260808 05:02:48.132122 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 9811
I20260808 05:02:48.132217 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:48.132417 3770 external_mini_cluster.cc:1428] Setting key 29343f93bdcb326ab9355f09d882da57
I20260808 05:02:48.132939 9933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5178a81d1e459e941846cbae8797ff" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41595 } }
I20260808 05:02:48.133148 9933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:48.133203 9933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f5178a81d1e459e941846cbae8797ff, State: Initialized, Role: FOLLOWER
I20260808 05:02:48.133358 9933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5178a81d1e459e941846cbae8797ff" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41595 } }
I20260808 05:02:48.133445 9933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:48.133483 9933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:48.133533 9933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:48.134236 9933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5178a81d1e459e941846cbae8797ff" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41595 } }
I20260808 05:02:48.134343 9933 leader_election.cc:302] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f5178a81d1e459e941846cbae8797ff; no voters:
I20260808 05:02:48.134491 9933 leader_election.cc:288] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:48.134531 9953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:48.134670 9953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [term 1 LEADER]: Becoming Leader. State: Replica: 8f5178a81d1e459e941846cbae8797ff, State: Running, Role: LEADER
I20260808 05:02:48.134745 9933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:48.134806 9953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5178a81d1e459e941846cbae8797ff" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41595 } }
I20260808 05:02:48.135171 9953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f5178a81d1e459e941846cbae8797ff. Latest consensus state: current_term: 1 leader_uuid: "8f5178a81d1e459e941846cbae8797ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5178a81d1e459e941846cbae8797ff" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41595 } } }
I20260808 05:02:48.135347 9953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:48.135215 9960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f5178a81d1e459e941846cbae8797ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5178a81d1e459e941846cbae8797ff" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41595 } } }
I20260808 05:02:48.135517 9960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:48.135639 9972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:48.136191 9972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:48.138507 9972 catalog_manager.cc:1347] Generated new cluster ID: b73c28315c004b35a10800263567cdf7
I20260808 05:02:48.138558 9972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 05:02:48 dist-test-slave-gjw7 krb5kdc[9781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:48.144065 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:48.132949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35659 (local address 127.3.174.190:41595)
0808 05:02:48.133152 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:48.133162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:48.133209 (+ 47us) server_negotiation.cc:407] Connection header received
0808 05:02:48.133364 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:48.133372 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:48.133431 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:48.133578 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:48.134235 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:48.135794 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:48.136869 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:48.137514 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:48.137617 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:48.140405 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:48.140410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:48.140410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:48.140423 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:48.142426 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:48.142806 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:48.142810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:48.142811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:48.142944 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:48.143126 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:48.143127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:48.143127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:48.143469 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:48.143633 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:48.143712 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 05:02:48.143859 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260808 05:02:48.145217 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:48.148557 9972 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:48.149035 9972 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:48.153752 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:48.146703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47035 (local address 127.3.174.190:41595)
0808 05:02:48.146764 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:48.146766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:48.146948 (+ 182us) server_negotiation.cc:407] Connection header received
0808 05:02:48.146995 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:48.146996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:48.147007 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:48.147056 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:48.147670 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:48.148312 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:48.149348 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:48.149647 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:48.149708 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:48.151050 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:48.151051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:48.151052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:48.151059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:48.152581 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:48.152968 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:48.152968 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:48.152969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:48.153022 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:48.153214 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:48.153214 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:48.153214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:48.153363 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:48.153448 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:48.153573 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 05:02:48.153661 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 05:02:48.155104 9972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f5178a81d1e459e941846cbae8797ff: Generated new TSK 0
I20260808 05:02:48.155279 9972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:48.160413 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 9811
2026-08-08T05:02:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:48 dist-test-slave-gjw7 krb5kdc[10003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:48 dist-test-slave-gjw7 krb5kdc[10003](info): set up 2 sockets
Aug 08 05:02:48 dist-test-slave-gjw7 krb5kdc[10003](info): commencing operation
krb5kdc: starting...
W20260808 05:02:49.421638 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:49 dist-test-slave-gjw7 krb5kdc[10003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:49.568210 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:36547
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:37897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:36547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:49.651942 10029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:49.652136 10029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:49.652158 10029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:49.653329 10029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:49.653373 10029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:49.653386 10029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:49.653398 10029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:49.653409 10029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:49.654887 10029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:37897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:36547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:36547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:49.655117 10029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:49.655512 10029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:49.660395 10037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:49.660516 10029 server_base.cc:1034] running on GCE node
W20260808 05:02:49.660403 10045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:49.660405 10038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:49.660880 10029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:49.661170 10029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:49.662374 10029 hybrid_clock.cc:648] HybridClock initialized: now 1786165369662350 us; error 39 us; skew 500 ppm
Aug 08 05:02:49 dist-test-slave-gjw7 krb5kdc[10003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165369, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:49.664752 10029 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:49.665149 10029 webserver.cc:460] Webserver started at http://127.3.174.190:38869/ using document root <none> and password file <none>
I20260808 05:02:49.665449 10029 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:49.665503 10029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:49.665627 10029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:49.666776 10029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "e943fe8bf87140ebb8046e728e8be657"
format_stamp: "Formatted at 2026-08-08 05:02:49 on dist-test-slave-gjw7"
server_key: "e0889baff3bf76fbba6c1423dc23e716"
server_key_iv: "de9ebae7e247c5eb9a223305739ec7f2"
server_key_version: "encryptionkey@0"
I20260808 05:02:49.667109 10029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "e943fe8bf87140ebb8046e728e8be657"
format_stamp: "Formatted at 2026-08-08 05:02:49 on dist-test-slave-gjw7"
server_key: "e0889baff3bf76fbba6c1423dc23e716"
server_key_iv: "de9ebae7e247c5eb9a223305739ec7f2"
server_key_version: "encryptionkey@0"
I20260808 05:02:49.668382 10029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:49.669113 10059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:49.669278 10029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:49.669394 10029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "e943fe8bf87140ebb8046e728e8be657"
format_stamp: "Formatted at 2026-08-08 05:02:49 on dist-test-slave-gjw7"
server_key: "e0889baff3bf76fbba6c1423dc23e716"
server_key_iv: "de9ebae7e247c5eb9a223305739ec7f2"
server_key_version: "encryptionkey@0"
I20260808 05:02:49.669459 10029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:49.682344 10029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:49.684015 10029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:49.684146 10029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:49.688014 10029 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:36547
I20260808 05:02:49.688045 10162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:36547 every 8 connection(s)
I20260808 05:02:49.688369 10029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:49.689256 10164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:49.691547 10164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657: Bootstrap starting.
I20260808 05:02:49.692111 10164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:49.692344 10164 log.cc:824] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:49.693003 10164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657: No bootstrap required, opened a new log
I20260808 05:02:49.693368 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 10029
I20260808 05:02:49.693487 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:49.693689 3770 external_mini_cluster.cc:1428] Setting key caa2b185d9955cd190463e09f609cd3c
I20260808 05:02:49.695087 10164 raft_consensus.cc:348] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e943fe8bf87140ebb8046e728e8be657" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36547 } }
I20260808 05:02:49.695240 10164 raft_consensus.cc:374] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:49.695274 10164 raft_consensus.cc:729] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e943fe8bf87140ebb8046e728e8be657, State: Initialized, Role: FOLLOWER
I20260808 05:02:49.695459 10164 consensus_queue.cc:260] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e943fe8bf87140ebb8046e728e8be657" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36547 } }
I20260808 05:02:49.695544 10164 raft_consensus.cc:388] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:49.695567 10164 raft_consensus.cc:482] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:49.695603 10164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:49.696130 10164 raft_consensus.cc:504] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e943fe8bf87140ebb8046e728e8be657" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36547 } }
I20260808 05:02:49.696226 10164 leader_election.cc:302] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e943fe8bf87140ebb8046e728e8be657; no voters:
I20260808 05:02:49.696383 10164 leader_election.cc:288] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:49.696404 10170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:49.696509 10170 raft_consensus.cc:686] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [term 1 LEADER]: Becoming Leader. State: Replica: e943fe8bf87140ebb8046e728e8be657, State: Running, Role: LEADER
I20260808 05:02:49.696640 10164 sys_catalog.cc:564] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:49.696723 10170 consensus_queue.cc:237] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e943fe8bf87140ebb8046e728e8be657" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36547 } }
I20260808 05:02:49.697089 10170 sys_catalog.cc:455] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e943fe8bf87140ebb8046e728e8be657. Latest consensus state: current_term: 1 leader_uuid: "e943fe8bf87140ebb8046e728e8be657" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e943fe8bf87140ebb8046e728e8be657" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36547 } } }
I20260808 05:02:49.697387 10170 sys_catalog.cc:458] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:49.697256 10171 sys_catalog.cc:455] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e943fe8bf87140ebb8046e728e8be657" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e943fe8bf87140ebb8046e728e8be657" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 36547 } } }
I20260808 05:02:49.697817 10171 sys_catalog.cc:458] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657 [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:49.697829 10186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:49.698498 10186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:49.700168 10169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:49.694180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39413 (local address 127.3.174.190:36547)
0808 05:02:49.694389 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:49.694399 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:49.694598 (+ 199us) server_negotiation.cc:407] Connection header received
0808 05:02:49.695051 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:49.695057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:49.695102 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:49.695208 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:49.695492 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:49.696935 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:49.698003 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:49.698579 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:49.698682 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:49.699404 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:49.699408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:49.699409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:49.699599 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:49.699640 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:49.699656 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:49.699715 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:49.699899 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 05:02:49.699965 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260808 05:02:49.700196 10186 catalog_manager.cc:1347] Generated new cluster ID: a23298dd13ca4aa8ac7b4b682dfde4ea
I20260808 05:02:49.700416 10186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:49.701124 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:49.708781 10169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:49.702595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49063 (local address 127.3.174.190:36547)
0808 05:02:49.702635 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:49.702637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:49.705452 (+ 2815us) server_negotiation.cc:407] Connection header received
0808 05:02:49.705544 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:49.705546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:49.705557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:49.705621 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:49.705749 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:49.706486 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:49.707450 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:49.707745 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:49.707813 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:49.708481 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:49.708482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:49.708482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:49.708489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:49.708505 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:49.708510 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:49.708559 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:49.708659 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 05:02:49.708682 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 05:02:49.708938 10186 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:49.709573 10186 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:49.714244 10186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e943fe8bf87140ebb8046e728e8be657: Generated new TSK 0
I20260808 05:02:49.714409 10186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:49.716676 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 10029
2026-08-08T05:02:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:49 dist-test-slave-gjw7 krb5kdc[10213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:49 dist-test-slave-gjw7 krb5kdc[10213](info): set up 2 sockets
Aug 08 05:02:49 dist-test-slave-gjw7 krb5kdc[10213](info): commencing operation
krb5kdc: starting...
W20260808 05:02:50.922503 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:51 dist-test-slave-gjw7 krb5kdc[10213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:51.044018 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:41795
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:33893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:41795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:51.127561 10233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:51.127769 10233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:51.127790 10233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:51.128981 10233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:51.129024 10233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:51.129037 10233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:51.129048 10233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:51.129069 10233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:51.130607 10233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:33893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:41795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:41795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:51.130859 10233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:51.131078 10233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:51.135901 10253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:51.136013 10248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:51.136170 10233 server_base.cc:1034] running on GCE node
W20260808 05:02:51.136307 10249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:51.136582 10233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:51.136881 10233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:51.138047 10233 hybrid_clock.cc:648] HybridClock initialized: now 1786165371138012 us; error 47 us; skew 500 ppm
Aug 08 05:02:51 dist-test-slave-gjw7 krb5kdc[10213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165371, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:51.140604 10233 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:51.141076 10233 webserver.cc:460] Webserver started at http://127.3.174.190:40603/ using document root <none> and password file <none>
I20260808 05:02:51.141323 10233 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:51.141384 10233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:51.141518 10233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:51.142557 10233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "bf6e096916d34f46a846f79fe7e1e29a"
format_stamp: "Formatted at 2026-08-08 05:02:51 on dist-test-slave-gjw7"
server_key: "9562d539855ad0426f21138e7af563e4"
server_key_iv: "e90c033532290d04a0ba254f36b71a78"
server_key_version: "encryptionkey@0"
I20260808 05:02:51.142891 10233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "bf6e096916d34f46a846f79fe7e1e29a"
format_stamp: "Formatted at 2026-08-08 05:02:51 on dist-test-slave-gjw7"
server_key: "9562d539855ad0426f21138e7af563e4"
server_key_iv: "e90c033532290d04a0ba254f36b71a78"
server_key_version: "encryptionkey@0"
I20260808 05:02:51.144188 10233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 05:02:51.144927 10266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:51.145077 10233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:51.145177 10233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "bf6e096916d34f46a846f79fe7e1e29a"
format_stamp: "Formatted at 2026-08-08 05:02:51 on dist-test-slave-gjw7"
server_key: "9562d539855ad0426f21138e7af563e4"
server_key_iv: "e90c033532290d04a0ba254f36b71a78"
server_key_version: "encryptionkey@0"
I20260808 05:02:51.145243 10233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:51.148494 10233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:51.150529 10233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:51.150656 10233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:51.154671 10233 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:41795
I20260808 05:02:51.154704 10368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:41795 every 8 connection(s)
I20260808 05:02:51.155019 10233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:51.156018 10369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:51.158423 10369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a: Bootstrap starting.
I20260808 05:02:51.159013 10369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:51.159106 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 10233
I20260808 05:02:51.159221 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:51.159322 10369 log.cc:824] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:51.159430 3770 external_mini_cluster.cc:1428] Setting key bf48ff13af70fa68450b39a450df49ce
I20260808 05:02:51.160132 10369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a: No bootstrap required, opened a new log
I20260808 05:02:51.161906 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:51.159972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34277 (local address 127.3.174.190:41795)
0808 05:02:51.160136 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:51.160146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:51.160176 (+ 30us) server_negotiation.cc:407] Connection header received
0808 05:02:51.160642 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:51.160650 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:51.160687 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:51.160801 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:51.161458 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:51.161461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:51.161462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:51.161465 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:51.161516 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:51.161527 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:51.161596 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:51.161659 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 05:02:51.161694 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260808 05:02:51.162874 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:51.162789 10369 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6e096916d34f46a846f79fe7e1e29a" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41795 } }
I20260808 05:02:51.162943 10369 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 05:02:51.162981 10369 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6e096916d34f46a846f79fe7e1e29a, State: Initialized, Role: FOLLOWER
I20260808 05:02:51.163108 10369 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6e096916d34f46a846f79fe7e1e29a" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41795 } }
I20260808 05:02:51.163198 10369 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 05:02:51.163232 10369 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 05:02:51.163288 10369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:51.163818 10369 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6e096916d34f46a846f79fe7e1e29a" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41795 } }
I20260808 05:02:51.163962 10369 leader_election.cc:302] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf6e096916d34f46a846f79fe7e1e29a; no voters:
I20260808 05:02:51.164150 10369 leader_election.cc:288] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 05:02:51.164199 10382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:51.164408 10369 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:51.164392 10382 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [term 1 LEADER]: Becoming Leader. State: Replica: bf6e096916d34f46a846f79fe7e1e29a, State: Running, Role: LEADER
I20260808 05:02:51.164580 10382 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6e096916d34f46a846f79fe7e1e29a" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41795 } }
I20260808 05:02:51.164938 10384 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf6e096916d34f46a846f79fe7e1e29a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6e096916d34f46a846f79fe7e1e29a" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41795 } } }
I20260808 05:02:51.165166 10384 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:51.165098 10385 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf6e096916d34f46a846f79fe7e1e29a. Latest consensus state: current_term: 1 leader_uuid: "bf6e096916d34f46a846f79fe7e1e29a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6e096916d34f46a846f79fe7e1e29a" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 41795 } } }
I20260808 05:02:51.165571 10385 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:51.165860 10402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:51.166055 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:51.164254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38211 (local address 127.3.174.190:41795)
0808 05:02:51.164286 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:51.164288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:51.164573 (+ 285us) server_negotiation.cc:407] Connection header received
0808 05:02:51.164693 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:51.164694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:51.164699 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:51.164724 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:51.165714 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:51.165716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:51.165716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 05:02:51.165725 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:51.165739 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 05:02:51.165743 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:51.165790 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:51.165983 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 05:02:51.165996 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 05:02:51.166563 10402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:51.168047 10402 catalog_manager.cc:1347] Generated new cluster ID: f0e9db9a34424dd6a62c747c37bc2aeb
I20260808 05:02:51.168100 10402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:51.179299 10402 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:51.179473 10402 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:51.181486 10402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf6e096916d34f46a846f79fe7e1e29a: Generated new TSK 0
I20260808 05:02:51.181671 10402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 05:02:51.191859 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 10233
2026-08-08T05:02:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1468 ms)
[----------] 18 tests from AuthTokenIssuingTest (26645 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:51 dist-test-slave-gjw7 krb5kdc[10426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:51 dist-test-slave-gjw7 krb5kdc[10426](info): set up 2 sockets
Aug 08 05:02:51 dist-test-slave-gjw7 krb5kdc[10426](info): commencing operation
krb5kdc: starting...
W20260808 05:02:52.417943 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:52 dist-test-slave-gjw7 krb5kdc[10426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:52.554354 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46739
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:46739,127.3.174.189:45137,127.3.174.188:34061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:52.642436 10449 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 05:02:52.642678 10449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:52.642702 10449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:52.642730 10449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:52.644042 10449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:52.644094 10449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:52.644112 10449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:52.644127 10449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:52.644142 10449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:52.645752 10449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:46739,127.3.174.189:45137,127.3.174.188:34061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:46739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:52.646010 10449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:52.646241 10449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:52.651583 10460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:52.651619 10458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:52.651579 10462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:52.651726 10449 server_base.cc:1034] running on GCE node
I20260808 05:02:52.652083 10449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:52.652374 10449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:52.653540 10449 hybrid_clock.cc:648] HybridClock initialized: now 1786165372653516 us; error 34 us; skew 500 ppm
Aug 08 05:02:52 dist-test-slave-gjw7 krb5kdc[10426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:52.656154 10449 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:52.656630 10449 webserver.cc:460] Webserver started at http://127.3.174.190:38333/ using document root <none> and password file <none>
I20260808 05:02:52.656874 10449 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:52.656920 10449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:52.657053 10449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:52.658257 10449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "406fc788740349fea51fd5927128a4ca"
format_stamp: "Formatted at 2026-08-08 05:02:52 on dist-test-slave-gjw7"
server_key: "7e6be94a066f3fe608ca1b523c28cf95"
server_key_iv: "eefc32eb0a7271b6a4189f458805ef81"
server_key_version: "encryptionkey@0"
I20260808 05:02:52.658620 10449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "406fc788740349fea51fd5927128a4ca"
format_stamp: "Formatted at 2026-08-08 05:02:52 on dist-test-slave-gjw7"
server_key: "7e6be94a066f3fe608ca1b523c28cf95"
server_key_iv: "eefc32eb0a7271b6a4189f458805ef81"
server_key_version: "encryptionkey@0"
I20260808 05:02:52.660099 10449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 05:02:52.660970 10474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:52.661233 10449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 05:02:52.661366 10449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "406fc788740349fea51fd5927128a4ca"
format_stamp: "Formatted at 2026-08-08 05:02:52 on dist-test-slave-gjw7"
server_key: "7e6be94a066f3fe608ca1b523c28cf95"
server_key_iv: "eefc32eb0a7271b6a4189f458805ef81"
server_key_version: "encryptionkey@0"
I20260808 05:02:52.661458 10449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:52.687155 10449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:52.689113 10449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:52.689236 10449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:52.693485 10449 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:46739
I20260808 05:02:52.693538 10565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:46739 every 8 connection(s)
I20260808 05:02:52.693886 10449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:52.694962 10567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:52.696650 10567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:52.701032 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 10449
I20260808 05:02:52.701169 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:52.701462 3770 external_mini_cluster.cc:1428] Setting key 5441c3602c4515cc22e031781602e5bf
Aug 08 05:02:52 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:52.707217 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.697404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46739 (local address 127.0.0.1:54518)
0808 05:02:52.697646 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.697676 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:52.698168 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:52.698509 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:52.698512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:52.698798 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:52.698959 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:52.698963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.700019 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.700020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:52.700735 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:52.700737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.701499 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.701500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:52.701581 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:52.702136 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:52.702145 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:52.703826 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:52.705665 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:52.705672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:52.705674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:52.705873 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:52.706185 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:52.706185 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:52.706186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:52.706271 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:52.706722 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:52.706727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:52.706787 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 05:02:52.706887 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 05:02:52.707031 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":95,"threads_started":1}
I20260808 05:02:52.707227 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.697435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.3.174.190:46739)
0808 05:02:52.697589 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:52.697598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:52.697760 (+ 162us) server_negotiation.cc:407] Connection header received
0808 05:02:52.698304 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:52.698307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:52.698365 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:52.698456 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:52.699052 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.699928 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.700834 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.701364 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.701481 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:52.703918 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:52.703926 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:52.703927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:52.703935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:52.705585 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:52.705985 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:52.705989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:52.705990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:52.706111 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:52.706331 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:52.706332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:52.706332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:52.706655 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:52.706749 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:52.706959 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 05:02:52.707119 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260808 05:02:52.708158 10567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:52.708520 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.708397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:45137 (local address 127.0.0.1:54600)
0808 05:02:52.708447 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.708496 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 05:02:52.708647 10477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
I20260808 05:02:52.709096 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.709003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:45137 (local address 127.0.0.1:54610)
0808 05:02:52.709050 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.709074 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 05:02:52.709244 10567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.189:45137: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.3.174.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab.
Entry for principal kudu/127.3.174.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab.
I20260808 05:02:52.729535 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.189:45137
--webserver_interface=127.3.174.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab
--principal=kudu/127.3.174.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:46739,127.3.174.189:45137,127.3.174.188:34061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:02:52.758524 10567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } attempt: 1
I20260808 05:02:52.759081 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.758941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:45137 (local address 127.0.0.1:54614)
0808 05:02:52.759013 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.759049 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 05:02:52.759451 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.759361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:45137 (local address 127.0.0.1:54628)
0808 05:02:52.759405 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.759431 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 05:02:52.759555 10567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.189:45137: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
W20260808 05:02:52.818045 10582 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 05:02:52.818325 10582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:52.818356 10582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:52.818383 10582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:52.819631 10582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:52.819686 10582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:52.819703 10582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:52.819728 10582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:52.819743 10582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:52.821475 10582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:46739,127.3.174.189:45137,127.3.174.188:34061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab
--principal=kudu/127.3.174.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.189:45137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.174.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:52.821780 10582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:52.822022 10582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 05:02:52.827761 10567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } attempt: 2
W20260808 05:02:52.828136 10589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:52.828413 10593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:52.828145 10588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:52.828715 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.828426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:45137 (local address 127.0.0.1:54634)
0808 05:02:52.828518 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.828558 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 05:02:52.829558 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.829232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:45137 (local address 127.0.0.1:54640)
0808 05:02:52.829491 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.829528 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
I20260808 05:02:52.829613 10582 server_base.cc:1034] running on GCE node
W20260808 05:02:52.829668 10567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.189:45137: Network error: Client connection negotiation failed: client connection to 127.3.174.189:45137: connect: Connection refused (error 111)
I20260808 05:02:52.829936 10582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:52.830273 10582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:52.831542 10582 hybrid_clock.cc:648] HybridClock initialized: now 1786165372831450 us; error 43 us; skew 500 ppm
Aug 08 05:02:52 dist-test-slave-gjw7 krb5kdc[10426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:52.835421 10582 init.cc:377] Logged in from keytab as kudu/127.3.174.189@KRBTEST.COM (short username kudu)
I20260808 05:02:52.836437 10582 webserver.cc:460] Webserver started at http://127.3.174.189:44279/ using document root <none> and password file <none>
I20260808 05:02:52.836848 10582 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:52.836978 10582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:52.837282 10582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:52.838932 10582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/data/instance:
uuid: "29f191ab3e374221a175d5192cf9450e"
format_stamp: "Formatted at 2026-08-08 05:02:52 on dist-test-slave-gjw7"
server_key: "b0f169aab0140746e924ca7c2b4045c7"
server_key_iv: "9c10ebea717330f9fbcb8f4ec0379bd0"
server_key_version: "encryptionkey@0"
I20260808 05:02:52.839838 10582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/wal/instance:
uuid: "29f191ab3e374221a175d5192cf9450e"
format_stamp: "Formatted at 2026-08-08 05:02:52 on dist-test-slave-gjw7"
server_key: "b0f169aab0140746e924ca7c2b4045c7"
server_key_iv: "9c10ebea717330f9fbcb8f4ec0379bd0"
server_key_version: "encryptionkey@0"
I20260808 05:02:52.842892 10582 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 05:02:52.844552 10614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:52.844874 10582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:52.845038 10582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/wal
uuid: "29f191ab3e374221a175d5192cf9450e"
format_stamp: "Formatted at 2026-08-08 05:02:52 on dist-test-slave-gjw7"
server_key: "b0f169aab0140746e924ca7c2b4045c7"
server_key_iv: "9c10ebea717330f9fbcb8f4ec0379bd0"
server_key_version: "encryptionkey@0"
I20260808 05:02:52.845239 10582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:52.875999 10582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:52.878345 10582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:52.878504 10582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:52.882655 10582 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.189:45137
I20260808 05:02:52.882694 10718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.189:45137 every 8 connection(s)
I20260808 05:02:52.883064 10582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
I20260808 05:02:52.884101 10719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:52.885917 10719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:52.886212 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 10582
I20260808 05:02:52.886323 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-1/wal/instance
I20260808 05:02:52.886554 3770 external_mini_cluster.cc:1428] Setting key 9adb43809a3e2d6cc30ee056016a6fed
Aug 08 05:02:52 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.174.188@KRBTEST.COM; defaulting to no policy
I20260808 05:02:52.895655 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.886771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54522 (local address 127.3.174.190:46739)
0808 05:02:52.886847 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:52.886849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:52.887108 (+ 259us) server_negotiation.cc:407] Connection header received
0808 05:02:52.887624 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:52.887625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:52.887644 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:52.887697 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:52.888402 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.888914 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.890021 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.890236 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.890298 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:52.892565 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:52.892567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:52.892568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:52.892579 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:52.894161 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:52.894606 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:52.894607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:52.894608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:52.894680 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:52.894936 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:52.894937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:52.894937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:52.895081 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:52.895143 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:52.895460 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 05:02:52.895555 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 05:02:52.895895 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.886782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46739 (local address 127.0.0.1:54522)
0808 05:02:52.886967 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.887010 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:52.887510 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:52.887769 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:52.887773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:52.888046 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:52.888289 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:52.888294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.889011 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.889013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:52.889919 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:52.889921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.890337 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.890338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:52.890473 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:52.890997 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:52.891016 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:52.892458 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:52.894251 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:52.894263 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:52.894266 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:52.894524 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:52.894742 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:52.894743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:52.894743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:52.894872 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:52.895143 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:52.895153 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:52.895323 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 05:02:52.895423 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 05:02:52.895606 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
I20260808 05:02:52.896592 10719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.188@KRBTEST.COM" created.
Aug 08 05:02:52 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:02:52.904798 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.896880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:45137 (local address 127.0.0.1:54654)
0808 05:02:52.896954 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.896966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:52.897016 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:52.897410 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:52.897411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:52.897539 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:52.897597 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:52.897598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.898669 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.898671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:52.899364 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:52.899368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.899848 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.899849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:52.899927 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:52.900431 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:52.900436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:52.901794 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:52.903540 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:52.903542 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:52.903542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:52.903793 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:52.903997 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:52.903997 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:52.903998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:52.904067 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:52.904441 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:52.904442 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:52.904496 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 05:02:52.904571 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 05:02:52.904690 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 05:02:52.904906 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.897032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54654 (local address 127.3.174.189:45137)
0808 05:02:52.897248 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:52.897252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:52.897270 (+ 18us) server_negotiation.cc:407] Connection header received
0808 05:02:52.897311 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:52.897314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:52.897329 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:52.897397 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:52.897659 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.898568 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.899452 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.899772 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.899829 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:52.901891 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:52.901901 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:52.901901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:52.901909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:52.903441 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:52.903872 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:52.903876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:52.903876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:52.903930 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:52.904160 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:52.904161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:52.904162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:52.904361 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:52.904440 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:52.904602 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 05:02:52.904784 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":163,"thread_start_us":118,"threads_started":1}
I20260808 05:02:52.905750 10719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:52.906117 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.905981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:36978)
0808 05:02:52.906032 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.906087 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 05:02:52.906246 10618 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
I20260808 05:02:52.906669 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.906595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:36980)
0808 05:02:52.906622 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.906649 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 05:02:52.906787 10719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:34061: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab.
Entry for principal kudu/127.3.174.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab.
I20260808 05:02:52.915925 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.188:34061
--webserver_interface=127.3.174.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab
--principal=kudu/127.3.174.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:46739,127.3.174.189:45137,127.3.174.188:34061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:02:52.920843 10567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } attempt: 3
Aug 08 05:02:52 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:02:52.927996 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.921395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.3.174.189:45137)
0808 05:02:52.921467 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:52.921469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:52.921479 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:52.921509 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:52.921510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:52.921521 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:52.921566 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:52.921911 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.922411 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.923144 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.923378 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.923438 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:52.925535 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:52.925537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:52.925537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:52.925546 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:52.926885 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:52.927293 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:52.927294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:52.927294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:52.927357 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:52.927541 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:52.927541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:52.927542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:52.927649 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:52.927698 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:52.927858 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 05:02:52.927925 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 05:02:52.928126 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.921276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:45137 (local address 127.0.0.1:54658)
0808 05:02:52.921364 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.921378 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:52.921434 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:52.921579 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:52.921580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:52.921751 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:52.921822 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:52.921825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.922496 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.922497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:52.923053 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:52.923056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:52.923451 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:52.923452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:52.923525 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:52.924050 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:52.924055 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:52.925432 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:52.926987 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:52.926988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:52.926989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:52.927222 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:52.927415 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:52.927416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:52.927416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:52.927488 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:52.927702 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:52.927703 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:52.927761 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 05:02:52.927837 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 05:02:52.927948 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 05:02:52.928578 10567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:52.928974 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.928860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:36982)
0808 05:02:52.928912 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.928948 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 05:02:52.929256 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.929179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:36988)
0808 05:02:52.929218 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.929241 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 05:02:52.929355 10567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:34061: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
I20260808 05:02:52.956089 10719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } attempt: 1
I20260808 05:02:52.956748 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.956575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37002)
0808 05:02:52.956645 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.956682 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 05:02:52.957229 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.957084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37008)
0808 05:02:52.957159 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.957199 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 05:02:52.957355 10719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:34061: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
I20260808 05:02:52.960525 10567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } attempt: 1
I20260808 05:02:52.961025 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.960881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37016)
0808 05:02:52.960949 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.960993 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 05:02:52.961380 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:52.961275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37026)
0808 05:02:52.961330 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:52.961358 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 05:02:52.961475 10567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:34061: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
W20260808 05:02:53.006850 10740 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 05:02:53.007093 10740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:53.007110 10740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:53.007133 10740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:53.008309 10740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:53.008349 10740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:53.008363 10740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:53.008374 10740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:53.008385 10740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:53.009951 10740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:46739,127.3.174.189:45137,127.3.174.188:34061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab
--principal=kudu/127.3.174.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.188:34061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.174.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:53.010166 10740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:53.010383 10740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:53.015311 10745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:53.015319 10747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:53.015496 10740 server_base.cc:1034] running on GCE node
W20260808 05:02:53.015341 10749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:53.015933 10740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:53.016261 10740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:53.017407 10740 hybrid_clock.cc:648] HybridClock initialized: now 1786165373017383 us; error 38 us; skew 500 ppm
Aug 08 05:02:53 dist-test-slave-gjw7 krb5kdc[10426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165373, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:53.019950 10740 init.cc:377] Logged in from keytab as kudu/127.3.174.188@KRBTEST.COM (short username kudu)
I20260808 05:02:53.020459 10740 webserver.cc:460] Webserver started at http://127.3.174.188:35505/ using document root <none> and password file <none>
I20260808 05:02:53.020704 10740 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:53.020756 10740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:53.020902 10740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:53.022089 10740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/data/instance:
uuid: "81299936d6b840d98e4d53421908ab0c"
format_stamp: "Formatted at 2026-08-08 05:02:53 on dist-test-slave-gjw7"
server_key: "499087217e7feef76319505c90fc613c"
server_key_iv: "8f193fce5dc50cd8b100e7891d446545"
server_key_version: "encryptionkey@0"
I20260808 05:02:53.022464 10740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/wal/instance:
uuid: "81299936d6b840d98e4d53421908ab0c"
format_stamp: "Formatted at 2026-08-08 05:02:53 on dist-test-slave-gjw7"
server_key: "499087217e7feef76319505c90fc613c"
server_key_iv: "8f193fce5dc50cd8b100e7891d446545"
server_key_version: "encryptionkey@0"
I20260808 05:02:53.023924 10740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 05:02:53.024631 10760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:53.024797 10740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 05:02:53.024887 10740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/wal
uuid: "81299936d6b840d98e4d53421908ab0c"
format_stamp: "Formatted at 2026-08-08 05:02:53 on dist-test-slave-gjw7"
server_key: "499087217e7feef76319505c90fc613c"
server_key_iv: "8f193fce5dc50cd8b100e7891d446545"
server_key_version: "encryptionkey@0"
I20260808 05:02:53.024952 10740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:53.025506 10719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } attempt: 2
I20260808 05:02:53.025890 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.025768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37040)
0808 05:02:53.025820 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:53.025864 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 05:02:53.026324 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.026200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37042)
0808 05:02:53.026271 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:53.026298 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 05:02:53.026428 10719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:34061: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
I20260808 05:02:53.036644 10567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } attempt: 2
I20260808 05:02:53.037236 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.037030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37046)
0808 05:02:53.037096 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:53.037133 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 05:02:53.037736 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.037638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37062)
0808 05:02:53.037682 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:53.037715 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 05:02:53.037849 10567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:34061: Network error: Client connection negotiation failed: client connection to 127.3.174.188:34061: connect: Connection refused (error 111)
I20260808 05:02:53.045536 10740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:53.047616 10740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:53.047775 10740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:53.052147 10740 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.188:34061
I20260808 05:02:53.052192 10862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.188:34061 every 8 connection(s)
I20260808 05:02:53.052565 10740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
I20260808 05:02:53.053678 10863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:53.055416 10863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 05:02:53 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165373, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:53.062619 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 10740
I20260808 05:02:53.062736 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786165096473501-3770-0/minicluster-data/master-2/wal/instance
I20260808 05:02:53.062970 3770 external_mini_cluster.cc:1428] Setting key 63baad0b5455c4dd49337a76bad64b16
I20260808 05:02:53.066128 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.056144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:46739 (local address 127.0.0.1:54532)
0808 05:02:53.056372 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:53.056415 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:53.056903 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:53.057210 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:53.057215 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:53.057565 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:53.057777 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.057782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.058456 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.058457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.059281 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.059283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.059703 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.059704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.059823 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.060315 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:53.060324 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:53.061783 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:53.064273 (+ 2490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.064281 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.064284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.064589 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.064795 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.064795 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.064796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.064952 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.065332 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:53.065341 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:53.065550 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 05:02:53.065670 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 05:02:53.065863 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":82,"threads_started":1}
I20260808 05:02:53.066628 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.056204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.3.174.190:46739)
0808 05:02:53.056288 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.056290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.056499 (+ 209us) server_negotiation.cc:407] Connection header received
0808 05:02:53.057019 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.057021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.057033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.057094 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.057866 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.058370 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.059392 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.059584 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.059640 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.061888 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.061890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.061890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.061899 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.064191 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.064668 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.064669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.064669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.064732 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.065065 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.065066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.065067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.065210 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.065273 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.066436 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.066529 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 05:02:53.067198 10863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 05:02:53 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
I20260808 05:02:53.072697 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.063408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.3.174.188:34061)
0808 05:02:53.063625 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.063633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.063669 (+ 36us) server_negotiation.cc:407] Connection header received
0808 05:02:53.063697 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.063700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.063715 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.063799 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.064481 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.065559 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.066593 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.067066 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.067137 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.070085 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.070094 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.070094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.070101 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.071550 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.071885 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.071889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.071889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.071943 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.072119 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.072120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.072120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.072277 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.072423 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.072502 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.072584 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":89,"threads_started":1}
Aug 08 05:02:53 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165373, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:02:53.080555 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.073403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54666 (local address 127.3.174.189:45137)
0808 05:02:53.073463 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.073465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.073475 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:53.073503 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.073504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.073515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.073576 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.073948 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.074703 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.075354 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.075680 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.075753 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.077578 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.077579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.077579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.077587 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.079265 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.079684 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.079685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.079686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.079752 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.080001 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.080003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.080003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.080137 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.080201 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.080377 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.080448 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 05:02:53.080564 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.067513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:45137 (local address 127.0.0.1:54666)
0808 05:02:53.067578 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:53.067590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:53.067632 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:53.073603 (+ 5971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:53.073605 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:53.073762 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:53.073845 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.073847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.074786 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.074787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.075281 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.075282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.075752 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.075752 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.075809 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.076240 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:53.076245 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:53.077494 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:53.079359 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.079360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.079360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.079582 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.079844 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.079845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.079845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.079911 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.080206 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:53.080207 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:53.080262 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 05:02:53.080329 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 05:02:53.080433 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 05:02:53.081321 10863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 05:02:53 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165373, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
Aug 08 05:02:53 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:53.089275 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.081590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37088)
0808 05:02:53.081631 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:53.081639 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:53.081672 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:53.081767 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:53.081768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:53.081896 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:53.081927 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.081928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.082871 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.082873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.083638 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.083640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.084012 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.084013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.084061 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.084467 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:53.084472 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:53.085983 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:53.088125 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.088126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.088126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.088315 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.088554 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.088555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.088555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.088624 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.088914 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:53.088915 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:53.088963 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 05:02:53.089032 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 05:02:53.089134 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 05:02:53.089704 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.081631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.3.174.188:34061)
0808 05:02:53.081659 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.081661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.081668 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:02:53.081713 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.081713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.081723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.081765 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.081992 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.082766 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.083725 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.083936 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.084000 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.086076 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.086078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.086078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.086087 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.088065 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.088405 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.088407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.088408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.088480 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.088693 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.088694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.088695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.088838 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.088913 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.089070 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.089198 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":17}
I20260808 05:02:53.090065 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.081125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54548 (local address 127.3.174.190:46739)
0808 05:02:53.081197 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.081200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.081208 (+ 8us) server_negotiation.cc:407] Connection header received
0808 05:02:53.081237 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.081238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.081248 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.081331 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.081999 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.082773 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.084684 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.084963 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.085024 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.087355 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.087356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.087356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.087364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.088871 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.089234 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.089236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.089236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.089342 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.089547 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.089548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.089549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.089665 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.089856 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.089893 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.089974 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 05:02:53.091509 10863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c: Bootstrap starting.
I20260808 05:02:53.092389 10863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:53.092676 10863 log.cc:824] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:53.093591 10863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c: No bootstrap required, opened a new log
I20260808 05:02:53.095767 10863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } }
I20260808 05:02:53.096025 10863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81299936d6b840d98e4d53421908ab0c, State: Initialized, Role: FOLLOWER
I20260808 05:02:53.096158 10863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } }
I20260808 05:02:53.096522 10869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.096583 10863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:53.096632 10869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 05:02:53 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
W20260808 05:02:53.098734 10884 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:02:53.098791 10884 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:02:53.099059 10740 master_runner.cc:186] Error getting master registration for 127.3.174.190:46739: OK, instance_id { permanent_uuid: "406fc788740349fea51fd5927128a4ca" instance_seqno: 1786165372692315 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 05:02:53.100746 10740 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 05:02:53.103235 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.090853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.3.174.189:45137)
0808 05:02:53.090899 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.090901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.093028 (+ 2127us) server_negotiation.cc:407] Connection header received
0808 05:02:53.093067 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.093068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.093081 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.093136 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.093666 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.094158 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.094806 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.095023 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.095080 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.097473 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.097475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.097475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.097484 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.102099 (+ 4615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.102475 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.102476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.102477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.102534 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.102711 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.102712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.102713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.102852 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.103024 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.103062 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.103142 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 05:02:53.114923 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:53.115978 10526 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "406fc788740349fea51fd5927128a4ca"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54548
I20260808 05:02:53.117519 10719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } attempt: 3
Aug 08 05:02:53 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
I20260808 05:02:53.125672 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.117816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37100)
0808 05:02:53.117880 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:53.117892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:53.117940 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:53.118219 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:53.118220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:53.118423 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:53.118482 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.118484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.119426 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.119428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.119987 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.119989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.120477 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.120478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.120542 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.121026 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:53.121032 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:53.122577 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:53.124465 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.124466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.124466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.124701 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.124906 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.124906 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.124907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.124962 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.125342 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:53.125344 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:53.125393 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 05:02:53.125455 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 05:02:53.125567 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 05:02:53.125672 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.118016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37100 (local address 127.3.174.188:34061)
0808 05:02:53.118081 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.118084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.118095 (+ 11us) server_negotiation.cc:407] Connection header received
0808 05:02:53.118126 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.118128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.118138 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.118205 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.118543 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.119333 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.120084 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.120401 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.120457 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.122669 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.122672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.122672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.122681 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.124394 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.124781 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.124783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.124783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.124843 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.125018 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.125018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.125019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.125245 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.125335 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.125493 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.125567 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 05:02:53.127769 10719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e: Bootstrap starting.
I20260808 05:02:53.128580 10719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:53.128902 10719 log.cc:824] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:53.129881 10719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e: No bootstrap required, opened a new log
I20260808 05:02:53.131907 10719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } }
I20260808 05:02:53.132061 10719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29f191ab3e374221a175d5192cf9450e, State: Initialized, Role: FOLLOWER
I20260808 05:02:53.132143 10719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } }
I20260808 05:02:53.132537 10719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:53.132500 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.132752 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [sys.catalog]: This master's current role is: FOLLOWER
W20260808 05:02:53.134829 10903 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:02:53.134965 10903 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:02:53.136968 10567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } attempt: 3
I20260808 05:02:53.134747 10582 master_runner.cc:186] Error getting master registration for 127.3.174.190:46739: OK, instance_id { permanent_uuid: "406fc788740349fea51fd5927128a4ca" instance_seqno: 1786165372692315 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 05:02:53.137635 10582 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 05:02:53 dist-test-slave-gjw7 krb5kdc[10426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165372, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
I20260808 05:02:53.145133 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.137275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:34061 (local address 127.0.0.1:37104)
0808 05:02:53.137363 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:53.137375 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:53.137418 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:53.137937 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:53.137939 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:53.138100 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:53.138145 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.138148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.139046 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.139047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.139553 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:53.139555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.140023 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.140024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:53.140097 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.140581 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:53.140587 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:53.142133 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:53.143968 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.143969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.143969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.144181 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.144372 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:53.144372 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:53.144373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:53.144424 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:53.144689 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:53.144690 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:53.144738 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 05:02:53.144794 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 05:02:53.145022 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 05:02:53.145583 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.137741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37104 (local address 127.3.174.188:34061)
0808 05:02:53.137799 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.137801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.137810 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:53.137837 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.137838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.137848 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.137899 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.138228 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.138977 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.139638 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.139942 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.140008 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.142241 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.142244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.142244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.142252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.143872 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.144255 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.144256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.144256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.144308 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.144484 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.144485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.144485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.144620 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.145348 (+ 728us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.145408 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.145488 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 05:02:53.147176 10567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca: Bootstrap starting.
I20260808 05:02:53.147975 10567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:53.148275 10567 log.cc:824] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:53.158293 10567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca: No bootstrap required, opened a new log
I20260808 05:02:53.159932 10567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } }
I20260808 05:02:53.160123 10567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406fc788740349fea51fd5927128a4ca, State: Initialized, Role: FOLLOWER
I20260808 05:02:53.160219 10567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } }
I20260808 05:02:53.160538 10906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.160673 10526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 05:02:53.160620 10567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:53.160732 10526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:53.160691 10906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:53.161727 10526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } }
I20260808 05:02:53.162463 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "406fc788740349fea51fd5927128a4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "29f191ab3e374221a175d5192cf9450e"
I20260808 05:02:53.162616 10675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:53.163606 10675 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406fc788740349fea51fd5927128a4ca in term 1.
I20260808 05:02:53.163970 10477 leader_election.cc:302] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29f191ab3e374221a175d5192cf9450e, 406fc788740349fea51fd5927128a4ca; no voters:
I20260808 05:02:53.164426 10815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "406fc788740349fea51fd5927128a4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "81299936d6b840d98e4d53421908ab0c"
I20260808 05:02:53.164565 10815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:53.165661 10815 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406fc788740349fea51fd5927128a4ca in term 1.
I20260808 05:02:53.165939 10526 leader_election.cc:288] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [CANDIDATE]: Term 1 election: Requested vote from peers 29f191ab3e374221a175d5192cf9450e (127.3.174.189:45137), 81299936d6b840d98e4d53421908ab0c (127.3.174.188:34061)
I20260808 05:02:53.167584 10906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:53.167900 10906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [term 1 LEADER]: Becoming Leader. State: Replica: 406fc788740349fea51fd5927128a4ca, State: Running, Role: LEADER
I20260808 05:02:53.168012 10906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } }
I20260808 05:02:53.168550 10906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 406fc788740349fea51fd5927128a4ca. Latest consensus state: current_term: 1 leader_uuid: "406fc788740349fea51fd5927128a4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.168632 10906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [sys.catalog]: This master's current role is: LEADER
W20260808 05:02:53.171415 10933 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:02:53.171474 10933 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:02:53.177707 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.169710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.3.174.190:46739)
0808 05:02:53.169760 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.169763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.170006 (+ 243us) server_negotiation.cc:407] Connection header received
0808 05:02:53.170112 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.170113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.170125 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.170154 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.171021 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.171680 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.172480 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.172763 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.172834 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.174700 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.174701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.174701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.174709 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.176388 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.176817 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.176818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.176819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.176880 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.177065 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.177066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.177066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.177198 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.177270 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.177518 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.177605 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 05:02:53.186149 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.170077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.3.174.189:45137)
0808 05:02:53.170118 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.170120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.178818 (+ 8698us) server_negotiation.cc:407] Connection header received
0808 05:02:53.178918 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.178920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.178933 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.178963 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.179604 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.180357 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.181376 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.181665 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.181704 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.183240 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.183241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.183242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.183249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.184848 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.185231 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.185232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.185232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.185311 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.185507 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.185508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.185508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.185634 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.185706 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.185960 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.186043 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 05:02:53.188051 10940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:53.188313 10940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:53.188977 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:53.170412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37116 (local address 127.3.174.188:34061)
0808 05:02:53.170575 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.170577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.181466 (+ 10889us) server_negotiation.cc:407] Connection header received
0808 05:02:53.181606 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.181607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.181619 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.181650 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.182416 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.183133 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.184141 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:53.184429 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:53.184473 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:53.185942 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:53.185944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:53.185944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:53.185951 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:53.187513 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.187906 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.187907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.187908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.187979 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:53.188230 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:53.188231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:53.188231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:53.188412 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:53.188524 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:53.188789 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 05:02:53.188871 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
W20260808 05:02:53.190156 10487 tablet.cc:2369] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 05:02:53.198753 10815 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [term 1 FOLLOWER]: Refusing update from remote peer 406fc788740349fea51fd5927128a4ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:53.199141 10675 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [term 1 FOLLOWER]: Refusing update from remote peer 406fc788740349fea51fd5927128a4ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:53.199566 10906 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:53.199771 10906 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:53.201579 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 406fc788740349fea51fd5927128a4ca. Latest consensus state: current_term: 1 leader_uuid: "406fc788740349fea51fd5927128a4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.201686 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:53.202363 10869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 406fc788740349fea51fd5927128a4ca. Latest consensus state: current_term: 1 leader_uuid: "406fc788740349fea51fd5927128a4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.202473 10869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:53.211007 10934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "406fc788740349fea51fd5927128a4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.211156 10934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:53.211306 10949 mvcc.cc:205] Tried to move back new op lower bound from 7316133368615567360 to 7316133368497045504. Current Snapshot: MvccSnapshot[applied={T|T < 7316133368615567360}]
I20260808 05:02:53.211864 10906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "406fc788740349fea51fd5927128a4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.211973 10906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:53.212374 10869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "406fc788740349fea51fd5927128a4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.212450 10869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:53.212321 10957 mvcc.cc:205] Tried to move back new op lower bound from 7316133368615567360 to 7316133368497045504. Current Snapshot: MvccSnapshot[applied={T|T < 7316133368615567360}]
I20260808 05:02:53.212657 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "406fc788740349fea51fd5927128a4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "406fc788740349fea51fd5927128a4ca" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 46739 } } peers { permanent_uuid: "29f191ab3e374221a175d5192cf9450e" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 45137 } } peers { permanent_uuid: "81299936d6b840d98e4d53421908ab0c" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 34061 } } }
I20260808 05:02:53.212749 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:53.221817 10940 catalog_manager.cc:1347] Generated new cluster ID: 581c7875f85a495a90d5a557fb5bae15
I20260808 05:02:53.221899 10940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:53.249116 10940 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:53.250007 10940 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:53.261934 10940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 406fc788740349fea51fd5927128a4ca: Generated new TSK 0
I20260808 05:02:53.262158 10940 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 05:02:53.284186 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.282513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.3.174.189:45137)
0808 05:02:53.282557 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.282559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.283480 (+ 921us) server_negotiation.cc:407] Connection header received
0808 05:02:53.283587 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.283588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.283601 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.283630 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.284112 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54690: BlockingRecv error: recv got EOF from 127.0.0.1:54690 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 05:02:53.284965 10867 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.282864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37122 (local address 127.3.174.188:34061)
0808 05:02:53.282916 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.282918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.284444 (+ 1526us) server_negotiation.cc:407] Connection header received
0808 05:02:53.284541 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.284542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.284553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.284579 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.284896 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37122: BlockingRecv error: recv got EOF from 127.0.0.1:37122 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 05:02:53.285713 10573 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.283203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54572 (local address 127.3.174.190:46739)
0808 05:02:53.283246 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.283248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.285188 (+ 1940us) server_negotiation.cc:407] Connection header received
0808 05:02:53.285283 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.285284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.285317 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.285347 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.285651 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54572: BlockingRecv error: recv got EOF from 127.0.0.1:54572 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 05:02:53.288259 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.287472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.3.174.189:45137)
0808 05:02:53.287512 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.287514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.287718 (+ 204us) server_negotiation.cc:407] Connection header received
0808 05:02:53.287821 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.287822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.287843 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.287883 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.288208 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54702: BlockingRecv error: recv got EOF from 127.0.0.1:54702 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 05:02:53.291425 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.290576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.3.174.189:45137)
0808 05:02:53.290611 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.290612 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.290867 (+ 255us) server_negotiation.cc:407] Connection header received
0808 05:02:53.290983 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.290984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.290997 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.291024 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.291373 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54718: BlockingRecv error: recv got EOF from 127.0.0.1:54718 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 05:02:53.295789 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.294805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54730 (local address 127.3.174.189:45137)
0808 05:02:53.294849 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.294851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.295120 (+ 269us) server_negotiation.cc:407] Connection header received
0808 05:02:53.295277 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.295278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.295292 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.295327 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.295727 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54730: BlockingRecv error: recv got EOF from 127.0.0.1:54730 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 05:02:53.302428 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.301339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54746 (local address 127.3.174.189:45137)
0808 05:02:53.301386 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.301389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.301728 (+ 339us) server_negotiation.cc:407] Connection header received
0808 05:02:53.301846 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.301847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.301889 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.301931 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.302355 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54746: BlockingRecv error: recv got EOF from 127.0.0.1:54746 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 05:02:53.312969 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.312278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54758 (local address 127.3.174.189:45137)
0808 05:02:53.312332 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.312334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.312346 (+ 12us) server_negotiation.cc:407] Connection header received
0808 05:02:53.312379 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.312380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.312392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.312469 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.312901 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54758: BlockingRecv error: recv got EOF from 127.0.0.1:54758 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 05:02:53.333050 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.331368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54774 (local address 127.3.174.189:45137)
0808 05:02:53.331445 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.331448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.331658 (+ 210us) server_negotiation.cc:407] Connection header received
0808 05:02:53.331881 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.331883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.331906 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.332026 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.332947 (+ 921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54774: BlockingRecv error: recv got EOF from 127.0.0.1:54774 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 05:02:53.366873 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.366120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.3.174.189:45137)
0808 05:02:53.366182 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.366184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.366360 (+ 176us) server_negotiation.cc:407] Connection header received
0808 05:02:53.366410 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.366411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.366425 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.366478 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.366809 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54778: BlockingRecv error: recv got EOF from 127.0.0.1:54778 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 05:02:53.432889 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.432269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54794 (local address 127.3.174.189:45137)
0808 05:02:53.432333 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.432335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.432405 (+ 70us) server_negotiation.cc:407] Connection header received
0808 05:02:53.432467 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.432467 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.432478 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.432540 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.432839 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54794: BlockingRecv error: recv got EOF from 127.0.0.1:54794 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 05:02:53.562752 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.562115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54804 (local address 127.3.174.189:45137)
0808 05:02:53.562201 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.562202 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.562276 (+ 74us) server_negotiation.cc:407] Connection header received
0808 05:02:53.562317 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.562319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.562331 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.562397 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.562698 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54804: BlockingRecv error: recv got EOF from 127.0.0.1:54804 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 05:02:53.820765 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:53.820086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54820 (local address 127.3.174.189:45137)
0808 05:02:53.820160 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:53.820162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:53.820248 (+ 86us) server_negotiation.cc:407] Connection header received
0808 05:02:53.820280 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:53.820281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:53.820292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:53.820348 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:53.820713 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54820: BlockingRecv error: recv got EOF from 127.0.0.1:54820 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 05:02:54.099305 10884 catalog_manager.cc:1259] Loaded cluster ID: 581c7875f85a495a90d5a557fb5bae15
I20260808 05:02:54.099386 10884 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c: loading cluster ID for follower catalog manager: success
I20260808 05:02:54.100222 10884 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c: acquiring CA information for follower catalog manager: success
I20260808 05:02:54.100587 10884 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 81299936d6b840d98e4d53421908ab0c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 05:02:54.135536 10903 catalog_manager.cc:1259] Loaded cluster ID: 581c7875f85a495a90d5a557fb5bae15
I20260808 05:02:54.135614 10903 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e: loading cluster ID for follower catalog manager: success
I20260808 05:02:54.136451 10903 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e: acquiring CA information for follower catalog manager: success
I20260808 05:02:54.136751 10903 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29f191ab3e374221a175d5192cf9450e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 05:02:54.336530 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:54.334059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54828 (local address 127.3.174.189:45137)
0808 05:02:54.334190 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:54.334192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:54.334202 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:54.334288 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:54.334289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:54.334389 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:54.334466 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:54.334595 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.335103 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.335851 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.336065 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.336121 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:54.336245 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:54.336368 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:54.336428 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:54.336444 (+ 16us) server_negotiation.cc:299] Negotiation successful
0808 05:02:54.336461 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260808 05:02:54.340451 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:54.338303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.3.174.188:34061)
0808 05:02:54.338408 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:54.338410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:54.338417 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:02:54.338454 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:54.338454 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:54.338491 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:54.338556 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:54.338626 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.339093 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.339827 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.340015 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.340060 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:54.340161 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:54.340263 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:54.340317 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:54.340357 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 05:02:54.340373 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":48,"threads_started":1}
I20260808 05:02:54.344776 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:54.342266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37140 (local address 127.3.174.188:34061)
0808 05:02:54.342303 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:54.342305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:54.342339 (+ 34us) server_negotiation.cc:407] Connection header received
0808 05:02:54.342395 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:54.342396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:54.342407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:54.342450 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:54.342629 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.343088 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.344058 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.344304 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.344364 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:54.344519 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:54.344554 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:54.344601 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:54.344685 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 05:02:54.344708 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 05:02:54.344808 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:54.342072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.3.174.189:45137)
0808 05:02:54.342117 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:54.342119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:54.342332 (+ 213us) server_negotiation.cc:407] Connection header received
0808 05:02:54.342425 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:54.342426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:54.342437 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:54.342462 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:54.342646 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.343371 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.344102 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.344383 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.344437 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:54.344569 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:54.344598 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:54.344643 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:54.344727 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 05:02:54.344748 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 05:02:54.345275 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:54.342030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54576 (local address 127.3.174.190:46739)
0808 05:02:54.342170 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:54.342172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:54.342181 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:54.342279 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:54.342280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:54.342332 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:54.342445 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:54.342533 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.343307 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.344289 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:54.344574 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:54.344781 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:54.344845 (+ 64us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:54.344945 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:54.345157 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:54.345175 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 05:02:54.345197 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":53,"threads_started":1}
I20260808 05:02:54.346771 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 10449
I20260808 05:02:54.367367 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 10582
I20260808 05:02:54.377382 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 10740
2026-08-08T05:02:54Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3193 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 05:02:54 dist-test-slave-gjw7 krb5kdc[11114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 05:02:54 dist-test-slave-gjw7 krb5kdc[11114](info): set up 2 sockets
Aug 08 05:02:54 dist-test-slave-gjw7 krb5kdc[11114](info): commencing operation
krb5kdc: starting...
W20260808 05:02:55.610882 3770 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/test-user.keytab.
Aug 08 05:02:55 dist-test-slave-gjw7 krb5kdc[11114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T05:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T05:02:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.174.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
Entry for principal kudu/127.3.174.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab.
I20260808 05:02:55.732307 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:43323
--webserver_interface=127.3.174.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:34393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:43323,127.3.174.189:33143,127.3.174.188:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 05:02:55.816131 11134 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 05:02:55.816359 11134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:55.816385 11134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:55.816411 11134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:55.817664 11134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:55.817705 11134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:55.817724 11134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:55.817741 11134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:55.817756 11134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:55.819309 11134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:34393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:43323,127.3.174.189:33143,127.3.174.188:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.190.keytab
--principal=kudu/127.3.174.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.190:43323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.174.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:55.819564 11134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:55.819800 11134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:55.824695 11143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:55.824867 11134 server_base.cc:1034] running on GCE node
W20260808 05:02:55.824714 11145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:55.824721 11142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:55.825248 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:55.825562 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:55.826709 11134 hybrid_clock.cc:648] HybridClock initialized: now 1786165375826694 us; error 32 us; skew 500 ppm
Aug 08 05:02:55 dist-test-slave-gjw7 krb5kdc[11114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165375, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:55.829183 11134 init.cc:377] Logged in from keytab as kudu/127.3.174.190@KRBTEST.COM (short username kudu)
I20260808 05:02:55.829655 11134 webserver.cc:460] Webserver started at http://127.3.174.190:45851/ using document root <none> and password file <none>
I20260808 05:02:55.829877 11134 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:55.829919 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:55.830013 11134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:55.831053 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/data/instance:
uuid: "ed3414e4fda142698243e1083a01d19e"
format_stamp: "Formatted at 2026-08-08 05:02:55 on dist-test-slave-gjw7"
server_key: "5e3d2a91cb7337a4d9215545ef3d637f"
server_key_iv: "234c304351f432827437b787b25acc0c"
server_key_version: "encryptionkey@0"
I20260808 05:02:55.831372 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/wal/instance:
uuid: "ed3414e4fda142698243e1083a01d19e"
format_stamp: "Formatted at 2026-08-08 05:02:55 on dist-test-slave-gjw7"
server_key: "5e3d2a91cb7337a4d9215545ef3d637f"
server_key_iv: "234c304351f432827437b787b25acc0c"
server_key_version: "encryptionkey@0"
I20260808 05:02:55.832609 11134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 05:02:55.833461 11153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:55.833637 11134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:55.833729 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/wal
uuid: "ed3414e4fda142698243e1083a01d19e"
format_stamp: "Formatted at 2026-08-08 05:02:55 on dist-test-slave-gjw7"
server_key: "5e3d2a91cb7337a4d9215545ef3d637f"
server_key_iv: "234c304351f432827437b787b25acc0c"
server_key_version: "encryptionkey@0"
I20260808 05:02:55.833813 11134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:55.849568 11134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:55.851724 11134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:55.851871 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:55.856350 11134 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.190:43323
I20260808 05:02:55.856375 11255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.190:43323 every 8 connection(s)
I20260808 05:02:55.856755 11134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/data/info.pb
I20260808 05:02:55.857661 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 11134
I20260808 05:02:55.857762 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-0/wal/instance
I20260808 05:02:55.857837 11256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:55.857971 3770 external_mini_cluster.cc:1428] Setting key 741700bbe1591d8ef30b7f6fc5174955
I20260808 05:02:55.859424 11256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 05:02:55 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165375, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.174.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.189@KRBTEST.COM" created.
I20260808 05:02:55.868945 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:55.860137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:43323 (local address 127.0.0.1:59008)
0808 05:02:55.860326 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:55.860365 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:55.860849 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:55.861114 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:55.861116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:55.861434 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:55.861595 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:55.861600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:55.862597 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:55.862597 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:55.863269 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:55.863271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:55.863750 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:55.863752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:55.863815 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:55.864349 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:55.864358 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:55.865966 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:55.867501 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:55.867508 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:55.867509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:55.867695 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:55.867943 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:55.867944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:55.867944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:55.868029 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:55.868379 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:55.868385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:55.868464 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 05:02:55.868562 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 05:02:55.868797 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":82,"thread_start_us":86,"threads_started":1}
I20260808 05:02:55.868944 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:55.860157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59008 (local address 127.3.174.190:43323)
0808 05:02:55.860335 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:55.860344 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:55.860457 (+ 113us) server_negotiation.cc:407] Connection header received
0808 05:02:55.861003 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:55.861006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:55.861026 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:55.861101 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:55.861689 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:55.862504 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:55.863348 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:55.863656 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:55.863751 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:55.866046 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:55.866054 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:55.866054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:55.866061 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:55.867427 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:55.867785 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:55.867788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:55.867788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:55.867878 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:55.868091 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:55.868092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:55.868092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:55.868311 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:55.868392 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:55.868617 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 05:02:55.868716 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260808 05:02:55.870901 11256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:55.871236 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:55.871111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:33143 (local address 127.0.0.1:46632)
0808 05:02:55.871157 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:55.871208 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 05:02:55.871423 11160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
I20260808 05:02:55.871860 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:55.871756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:33143 (local address 127.0.0.1:46634)
0808 05:02:55.871806 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:55.871838 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 05:02:55.871970 11256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.189:33143: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab.
Entry for principal kudu/127.3.174.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab.
I20260808 05:02:55.890029 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.189:33143
--webserver_interface=127.3.174.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab
--principal=kudu/127.3.174.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:34393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:43323,127.3.174.189:33143,127.3.174.188:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 05:02:55.921204 11256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } attempt: 1
I20260808 05:02:55.921725 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:55.921595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:33143 (local address 127.0.0.1:46640)
0808 05:02:55.921661 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:55.921695 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 05:02:55.922091 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:55.921993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:33143 (local address 127.0.0.1:46652)
0808 05:02:55.922044 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:55.922069 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 05:02:55.922183 11256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.189:33143: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
I20260808 05:02:55.990353 11256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } attempt: 2
I20260808 05:02:55.991156 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:55.990860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:33143 (local address 127.0.0.1:46660)
0808 05:02:55.991002 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:55.991085 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260808 05:02:55.991046 11270 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 05:02:55.991550 11270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:55.991617 11270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:55.991688 11270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 05:02:55.991762 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:55.991563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:33143 (local address 127.0.0.1:46662)
0808 05:02:55.991668 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:55.991721 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 05:02:55.991920 11256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.189:33143: Network error: Client connection negotiation failed: client connection to 127.3.174.189:33143: connect: Connection refused (error 111)
W20260808 05:02:55.993614 11270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:55.993693 11270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:55.993721 11270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:55.993747 11270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:55.993786 11270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:55.996234 11270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:34393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:43323,127.3.174.189:33143,127.3.174.188:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.189.keytab
--principal=kudu/127.3.174.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.189:33143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.174.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:55.996627 11270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:55.996981 11270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:56.003386 11275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:56.003553 11270 server_base.cc:1034] running on GCE node
W20260808 05:02:56.003391 11279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:56.003397 11277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:56.003875 11270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:56.004145 11270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:56.005331 11270 hybrid_clock.cc:648] HybridClock initialized: now 1786165376005310 us; error 36 us; skew 500 ppm
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165376, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:56.007786 11270 init.cc:377] Logged in from keytab as kudu/127.3.174.189@KRBTEST.COM (short username kudu)
I20260808 05:02:56.008261 11270 webserver.cc:460] Webserver started at http://127.3.174.189:46359/ using document root <none> and password file <none>
I20260808 05:02:56.008466 11270 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:56.008517 11270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:56.008648 11270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:56.009797 11270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/data/instance:
uuid: "14415239f48b4717a8eaaee259707cff"
format_stamp: "Formatted at 2026-08-08 05:02:56 on dist-test-slave-gjw7"
server_key: "180c024555984589ce2e1659e8561260"
server_key_iv: "a64a0ebacf91bb3d998ea7870e1ba66b"
server_key_version: "encryptionkey@0"
I20260808 05:02:56.010134 11270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/wal/instance:
uuid: "14415239f48b4717a8eaaee259707cff"
format_stamp: "Formatted at 2026-08-08 05:02:56 on dist-test-slave-gjw7"
server_key: "180c024555984589ce2e1659e8561260"
server_key_iv: "a64a0ebacf91bb3d998ea7870e1ba66b"
server_key_version: "encryptionkey@0"
I20260808 05:02:56.011396 11270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 05:02:56.012076 11291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:56.012245 11270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 05:02:56.012332 11270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/wal
uuid: "14415239f48b4717a8eaaee259707cff"
format_stamp: "Formatted at 2026-08-08 05:02:56 on dist-test-slave-gjw7"
server_key: "180c024555984589ce2e1659e8561260"
server_key_iv: "a64a0ebacf91bb3d998ea7870e1ba66b"
server_key_version: "encryptionkey@0"
I20260808 05:02:56.012394 11270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:56.049636 11270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:56.051539 11270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:56.051689 11270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:56.055840 11270 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.189:33143
I20260808 05:02:56.055922 11377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.189:33143 every 8 connection(s)
I20260808 05:02:56.056304 11270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/data/info.pb
I20260808 05:02:56.056623 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 11270
I20260808 05:02:56.056741 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-1/wal/instance
I20260808 05:02:56.056959 3770 external_mini_cluster.cc:1428] Setting key 3226286f7fb26fa3e4043c73c27c384a
I20260808 05:02:56.057472 11379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:56.059077 11379 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.174.188@KRBTEST.COM; defaulting to no policy
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165376, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.174.188@KRBTEST.COM" created.
I20260808 05:02:56.068073 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.059792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59016 (local address 127.3.174.190:43323)
0808 05:02:56.059850 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.059852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.060053 (+ 201us) server_negotiation.cc:407] Connection header received
0808 05:02:56.060586 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.060587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.060604 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.060660 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.061362 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.061897 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.062879 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.063087 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.063144 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.065197 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.065199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.065199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.065206 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.066606 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.067023 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.067025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.067025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.067079 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.067347 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.067348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.067349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.067496 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.067558 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.067890 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.067983 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 05:02:56.068233 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.059752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:43323 (local address 127.0.0.1:59016)
0808 05:02:56.059923 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.059962 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:56.060477 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:56.060741 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:56.060745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:56.061006 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:56.061259 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.061264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.061990 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.061991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.062795 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.062797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.063174 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.063175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.063297 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.063822 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:56.063830 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:56.065106 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:56.066705 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.066713 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.066716 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.066949 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.067139 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.067140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.067140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.067253 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.067556 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:56.067564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:56.067743 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 05:02:56.067855 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 05:02:56.067984 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260808 05:02:56.068832 11379 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165376, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:02:56.076432 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.069078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:33143 (local address 127.0.0.1:46678)
0808 05:02:56.069121 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.069132 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:56.069169 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:56.069462 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:56.069464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:56.069579 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:56.069626 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.069627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.070762 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.070763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.071517 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.071520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.072063 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.072065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.072152 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.072522 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:56.072528 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:56.073877 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:56.075472 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.075473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.075473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.075667 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.075845 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.075845 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.075846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.075921 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.076161 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:56.076162 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:56.076216 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 05:02:56.076285 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 05:02:56.076365 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 05:02:56.076529 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.069145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.3.174.189:33143)
0808 05:02:56.069309 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.069315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.069335 (+ 20us) server_negotiation.cc:407] Connection header received
0808 05:02:56.069356 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.069358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.069373 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.069446 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.069695 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.070676 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.071603 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.071904 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.071958 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.073958 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.073966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.073966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.073972 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.075382 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.075742 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.075745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.075745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.075793 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.075982 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.075982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.075982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.076101 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.076155 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.076316 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.076414 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260808 05:02:56.077088 11379 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:56.077446 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.077347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41392)
0808 05:02:56.077379 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.077424 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 05:02:56.077564 11295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
I20260808 05:02:56.078023 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.077927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41398)
0808 05:02:56.077975 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.078001 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 05:02:56.078133 11379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:32897: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
I20260808 05:02:56.083076 11256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.174.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab.
Entry for principal kudu/127.3.174.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab.
I20260808 05:02:56.084125 3770 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskihVxiI/build/release/bin/kudu
/tmp/dist-test-taskihVxiI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.188:32897
--webserver_interface=127.3.174.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab
--principal=kudu/127.3.174.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.174.148:34393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.174.190:43323,127.3.174.189:33143,127.3.174.188:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165375, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:02:56.090299 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.083461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46682 (local address 127.3.174.189:33143)
0808 05:02:56.083515 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.083516 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.083525 (+ 9us) server_negotiation.cc:407] Connection header received
0808 05:02:56.083547 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.083548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.083559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.083603 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.083972 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.084556 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.085455 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.085662 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.085739 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.087575 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.087576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.087577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.087584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.088941 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.089520 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.089522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.089522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.089590 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.089784 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.089785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.089786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.089898 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.089962 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.090113 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.090195 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 05:02:56.090302 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.083401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:33143 (local address 127.0.0.1:46682)
0808 05:02:56.083444 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.083455 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:56.083499 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:56.083607 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:56.083608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:56.083809 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:56.083887 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.083890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.084641 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.084642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.085378 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.085383 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.085752 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.085753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.085830 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.086258 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:56.086263 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:56.087487 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:56.089219 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.089220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.089220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.089448 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.089652 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.089653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.089654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.089722 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.089962 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:56.089963 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:56.090014 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 05:02:56.090083 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 05:02:56.090197 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 05:02:56.090678 11256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:56.091034 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.090932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41400)
0808 05:02:56.090982 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.091013 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 05:02:56.091378 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.091288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41414)
0808 05:02:56.091328 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.091354 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 05:02:56.091471 11256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:32897: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
I20260808 05:02:56.122628 11256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } attempt: 1
I20260808 05:02:56.123162 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.122990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41422)
0808 05:02:56.123067 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.123122 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 05:02:56.123564 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.123466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41424)
0808 05:02:56.123512 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.123541 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 05:02:56.123675 11256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:32897: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
I20260808 05:02:56.127372 11379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } attempt: 1
I20260808 05:02:56.127770 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.127654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41440)
0808 05:02:56.127709 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.127743 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 05:02:56.128078 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.127996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41452)
0808 05:02:56.128036 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.128061 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 05:02:56.128157 11379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:32897: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
W20260808 05:02:56.171430 11388 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 05:02:56.171656 11388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 05:02:56.171675 11388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 05:02:56.171695 11388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 05:02:56.172880 11388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 05:02:56.172924 11388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 05:02:56.172937 11388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 05:02:56.172948 11388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 05:02:56.172959 11388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 05:02:56.174525 11388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.174.148:34393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.174.190:43323,127.3.174.189:33143,127.3.174.188:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/krb5kdc/kudu_127.3.174.188.keytab
--principal=kudu/127.3.174.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.174.188:32897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.174.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 04:08:35 UTC on ccycloud.quasar-ozjjgu.root.comops.site
build id 4574102
I20260808 05:02:56.174778 11388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 05:02:56.175043 11388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 05:02:56.179735 11405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:56.179755 11398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 05:02:56.179697 11397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 05:02:56.180924 11388 server_base.cc:1034] running on GCE node
I20260808 05:02:56.181161 11388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 05:02:56.181495 11388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 05:02:56.182655 11388 hybrid_clock.cc:648] HybridClock initialized: now 1786165376182634 us; error 36 us; skew 500 ppm
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165376, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 05:02:56.185056 11388 init.cc:377] Logged in from keytab as kudu/127.3.174.188@KRBTEST.COM (short username kudu)
I20260808 05:02:56.185582 11388 webserver.cc:460] Webserver started at http://127.3.174.188:36583/ using document root <none> and password file <none>
I20260808 05:02:56.185808 11388 fs_manager.cc:359] Metadata directory not provided
I20260808 05:02:56.185859 11388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 05:02:56.185992 11388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 05:02:56.187114 11388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/data/instance:
uuid: "97e85e4e203f4848a880035f4714ca8e"
format_stamp: "Formatted at 2026-08-08 05:02:56 on dist-test-slave-gjw7"
server_key: "db89a1e68bfe85befc345ba86c3d1f6f"
server_key_iv: "6f315504b7bb0c9f481bd7f37f9c3812"
server_key_version: "encryptionkey@0"
I20260808 05:02:56.187472 11388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/wal/instance:
uuid: "97e85e4e203f4848a880035f4714ca8e"
format_stamp: "Formatted at 2026-08-08 05:02:56 on dist-test-slave-gjw7"
server_key: "db89a1e68bfe85befc345ba86c3d1f6f"
server_key_iv: "6f315504b7bb0c9f481bd7f37f9c3812"
server_key_version: "encryptionkey@0"
I20260808 05:02:56.188751 11388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 05:02:56.189589 11420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 05:02:56.189807 11388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 05:02:56.189896 11388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/data,/tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/wal
uuid: "97e85e4e203f4848a880035f4714ca8e"
format_stamp: "Formatted at 2026-08-08 05:02:56 on dist-test-slave-gjw7"
server_key: "db89a1e68bfe85befc345ba86c3d1f6f"
server_key_iv: "6f315504b7bb0c9f481bd7f37f9c3812"
server_key_version: "encryptionkey@0"
I20260808 05:02:56.189975 11388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 05:02:56.196310 11379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } attempt: 2
I20260808 05:02:56.196781 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.196637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41468)
0808 05:02:56.196703 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.196749 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 05:02:56.197141 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.197055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41480)
0808 05:02:56.197096 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.197121 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 05:02:56.197268 11379 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:32897: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
I20260808 05:02:56.198825 11256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } attempt: 2
I20260808 05:02:56.199191 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.199081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41486)
0808 05:02:56.199131 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.199165 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 05:02:56.199663 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.199564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41496)
0808 05:02:56.199610 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.199637 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 05:02:56.199776 11256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.174.188:32897: Network error: Client connection negotiation failed: client connection to 127.3.174.188:32897: connect: Connection refused (error 111)
I20260808 05:02:56.199981 11388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 05:02:56.201988 11388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 05:02:56.202098 11388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 05:02:56.206171 11388 rpc_server.cc:307] RPC server started. Bound to: 127.3.174.188:32897
I20260808 05:02:56.206202 11503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.174.188:32897 every 8 connection(s)
I20260808 05:02:56.206552 11388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/data/info.pb
I20260808 05:02:56.207556 11504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 05:02:56.209188 11504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:56.210352 3770 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskihVxiI/build/release/bin/kudu as pid 11388
I20260808 05:02:56.210495 3770 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskihVxiI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786165096473501-3770-0/minicluster-data/master-2/wal/instance
I20260808 05:02:56.210733 3770 external_mini_cluster.cc:1428] Setting key f1a38bcca1d4af94d61e718246173545
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165376, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
I20260808 05:02:56.219715 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.210054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59022 (local address 127.3.174.190:43323)
0808 05:02:56.210172 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.210175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.210407 (+ 232us) server_negotiation.cc:407] Connection header received
0808 05:02:56.211034 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.211035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.211050 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.211119 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.211816 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.212493 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.213756 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.214036 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.214121 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.216669 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.216670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.216671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.216679 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.218209 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.218659 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.218660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.218661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.218717 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.219011 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.219012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.219012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.219141 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.219206 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.219544 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.219620 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 05:02:56.219851 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.210023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.190:43323 (local address 127.0.0.1:59022)
0808 05:02:56.210217 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.210299 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:56.210919 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:56.211256 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:56.211261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:56.211530 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:56.211723 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.211728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.212571 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.212573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.213671 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.213681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.214162 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.214164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.214288 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.214798 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:56.214808 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:56.216578 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:56.218293 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.218301 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.218303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.218577 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.218787 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.218788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.218788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.218920 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.219215 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:56.219221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:56.219387 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 05:02:56.219473 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 05:02:56.219621 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260808 05:02:56.220623 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.211141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.3.174.188:32897)
0808 05:02:56.211329 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.211334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.211423 (+ 89us) server_negotiation.cc:407] Connection header received
0808 05:02:56.211488 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.211491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.211523 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.211598 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.212376 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.213619 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.214598 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.214923 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.214988 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.217690 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.217698 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.217698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.217705 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.219232 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.219542 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.219545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.219545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.219605 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.219909 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.219910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.219911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.220082 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.220348 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.220445 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.220528 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260808 05:02:56.220707 11504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165376, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.190@KRBTEST.COM
I20260808 05:02:56.227552 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.220981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.189:33143 (local address 127.0.0.1:46696)
0808 05:02:56.221114 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.221124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:56.221175 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:56.221331 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:56.221332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:56.221473 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:56.221535 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.221537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.222210 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.222212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.222737 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.222739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.223071 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.223072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.223144 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.223534 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:56.223538 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:56.224913 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:56.226491 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.226493 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.226493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.226688 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.226911 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.226912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.226912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.226983 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.227259 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:56.227259 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:56.227300 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 05:02:56.227356 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 05:02:56.227450 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 05:02:56.227571 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.221041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.3.174.189:33143)
0808 05:02:56.221099 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.221101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.221179 (+ 78us) server_negotiation.cc:407] Connection header received
0808 05:02:56.221231 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.221232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.221242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.221323 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.221606 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.222133 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.222823 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.223012 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.223058 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.224996 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.224997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.224997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.225004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.226412 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.226751 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.226771 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.226772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.226833 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.227043 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.227044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.227044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.227184 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.227253 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.227403 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.227471 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 05:02:56.228044 11504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } has no permanent_uuid. Determining permanent_uuid...
I20260808 05:02:56.229208 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.221480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59024 (local address 127.3.174.190:43323)
0808 05:02:56.221511 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.221513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.221520 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:02:56.221596 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.221597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.221605 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.221655 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.222055 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.222544 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.223330 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.223599 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.223649 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.226634 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.226635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.226635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.226641 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.228170 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.228510 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.228511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.228511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.228565 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.228734 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.228735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.228735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.228847 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.229016 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.229050 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.229121 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165376, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.188@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.174.189@KRBTEST.COM
I20260808 05:02:56.234998 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.228288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41514)
0808 05:02:56.228350 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.228361 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:56.228412 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:56.228540 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:56.228541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:56.228659 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:56.228686 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.228688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.229349 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.229350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.230021 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.230024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.230496 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.230497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.230540 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.230858 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:56.230863 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:56.232183 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:56.233883 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.233885 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.233885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.234101 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.234342 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.234343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.234343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.234409 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.234682 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:56.234683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:56.234724 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 05:02:56.234805 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 05:02:56.234891 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 05:02:56.235071 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.228328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41514 (local address 127.3.174.188:32897)
0808 05:02:56.228359 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.228361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.228396 (+ 35us) server_negotiation.cc:407] Connection header received
0808 05:02:56.228470 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.228471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.228480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.228526 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.228760 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.229245 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.230160 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.230392 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.230452 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.232258 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.232260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.232260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.232267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.233785 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.234197 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.234198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.234199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.234264 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.234460 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.234461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.234462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.234596 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.234667 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.234855 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.234942 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":16}
I20260808 05:02:56.236708 11504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e: Bootstrap starting.
I20260808 05:02:56.237543 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.229832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.3.174.189:33143)
0808 05:02:56.229873 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.229875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.229885 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:56.229907 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.229908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.229916 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.229970 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.230444 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.231097 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.232005 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.232264 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.232337 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.234795 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.234796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.234797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.234803 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.236400 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.236700 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.236701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.236701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.236757 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.236963 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.236964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.236964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.237131 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.237348 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.237387 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.237463 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 05:02:56.237766 11504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:56.238090 11504 log.cc:824] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:56.238910 11504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e: No bootstrap required, opened a new log
I20260808 05:02:56.239035 3770 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 05:02:56.239894 11204 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ed3414e4fda142698243e1083a01d19e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59024
I20260808 05:02:56.240525 11504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } }
I20260808 05:02:56.240653 11504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e85e4e203f4848a880035f4714ca8e, State: Initialized, Role: FOLLOWER
I20260808 05:02:56.240746 11504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } }
I20260808 05:02:56.241053 11504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:56.241026 11511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.241122 11511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [sys.catalog]: This master's current role is: FOLLOWER
W20260808 05:02:56.242758 11540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:02:56.242815 11540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:02:56.243040 11388 master_runner.cc:186] Error getting master registration for 127.3.174.190:43323: OK, instance_id { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" instance_seqno: 1786165375855010 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 05:02:56.245048 11388 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 05:02:56.288461 11379 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } attempt: 3
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165376, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.189@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
I20260808 05:02:56.296558 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.288883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41524)
0808 05:02:56.288957 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.288972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:56.289039 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:56.289165 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:56.289165 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:56.289392 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:56.289445 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.289447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.290368 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.290369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.291156 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.291159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.291647 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.291647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.291703 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.292177 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:56.292182 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:56.293728 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:56.295447 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.295449 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.295449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.295682 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.295871 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.295871 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.295872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.295944 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.296246 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:56.296247 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:56.296293 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 05:02:56.296348 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 05:02:56.296447 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 05:02:56.296564 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.288964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41524 (local address 127.3.174.188:32897)
0808 05:02:56.289031 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.289033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.289045 (+ 12us) server_negotiation.cc:407] Connection header received
0808 05:02:56.289083 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.289084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.289095 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.289153 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.289517 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.290282 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.291260 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.291575 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.291645 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.293808 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.293810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.293810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.293819 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.295365 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.295752 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.295752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.295753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.295813 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.295996 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.295997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.295997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.296186 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.296311 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.296382 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.296465 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 05:02:56.298698 11379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff: Bootstrap starting.
I20260808 05:02:56.298925 11256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } attempt: 3
I20260808 05:02:56.299503 11379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:56.299831 11379 log.cc:824] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:56.300715 11379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff: No bootstrap required, opened a new log
I20260808 05:02:56.302887 11379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } }
I20260808 05:02:56.303043 11379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14415239f48b4717a8eaaee259707cff, State: Initialized, Role: FOLLOWER
I20260808 05:02:56.303135 11379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } }
Aug 08 05:02:56 dist-test-slave-gjw7 krb5kdc[11114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786165375, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.174.190@KRBTEST.COM for kudu/127.3.174.188@KRBTEST.COM
I20260808 05:02:56.303495 11379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:56.303573 11543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.303727 11543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [sys.catalog]: This master's current role is: FOLLOWER
W20260808 05:02:56.305482 11561 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:02:56.305605 11561 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:02:56.307139 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.299193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.174.188:32897 (local address 127.0.0.1:41534)
0808 05:02:56.299249 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 05:02:56.299259 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 05:02:56.299298 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 05:02:56.299415 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 05:02:56.299416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 05:02:56.299571 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 05:02:56.299626 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.299629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.300435 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.300437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.301250 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 05:02:56.301253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.301754 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.301755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 05:02:56.301814 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.302301 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 05:02:56.302306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 05:02:56.303743 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 05:02:56.305952 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.305953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.305953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.306191 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.306424 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 05:02:56.306425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 05:02:56.306425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 05:02:56.306488 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 05:02:56.306805 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 05:02:56.306806 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 05:02:56.306857 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 05:02:56.306924 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 05:02:56.307038 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 05:02:56.307536 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.299265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.3.174.188:32897)
0808 05:02:56.299297 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.299299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.299306 (+ 7us) server_negotiation.cc:407] Connection header received
0808 05:02:56.299357 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.299357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.299367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.299409 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.299696 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.300350 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.301352 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.301669 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.301741 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.304047 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.304049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.304049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.304057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.305845 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.306285 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.306287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.306287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.306351 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.306559 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.306559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.306560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.306710 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.306773 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.307362 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.307447 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 05:02:56.305908 11270 master_runner.cc:186] Error getting master registration for 127.3.174.190:43323: OK, instance_id { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" instance_seqno: 1786165375855010 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 05:02:56.308986 11270 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 05:02:56.310571 11256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e: Bootstrap starting.
I20260808 05:02:56.311359 11256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e: Neither blocks nor log segments found. Creating new log.
I20260808 05:02:56.311690 11256 log.cc:824] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e: Log is configured to *not* fsync() on all Append() calls
I20260808 05:02:56.312595 11256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e: No bootstrap required, opened a new log
I20260808 05:02:56.314082 11256 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } }
I20260808 05:02:56.314230 11256 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed3414e4fda142698243e1083a01d19e, State: Initialized, Role: FOLLOWER
I20260808 05:02:56.314312 11256 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } }
I20260808 05:02:56.314620 11564 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.314662 11256 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 05:02:56.314851 11204 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 05:02:56.314920 11204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:56.315773 11204 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } }
I20260808 05:02:56.316063 11204 leader_election.cc:288] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [CANDIDATE]: Term 1 election: Requested vote from peers 14415239f48b4717a8eaaee259707cff (127.3.174.189:33143), 97e85e4e203f4848a880035f4714ca8e (127.3.174.188:32897)
I20260808 05:02:56.316401 11564 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:56.316439 11333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ed3414e4fda142698243e1083a01d19e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "14415239f48b4717a8eaaee259707cff"
I20260808 05:02:56.316594 11333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [term 0 FOLLOWER]: Advancing to term 1
W20260808 05:02:56.317225 11587 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 05:02:56.317268 11587 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 05:02:56.317971 11333 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed3414e4fda142698243e1083a01d19e in term 1.
I20260808 05:02:56.317996 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ed3414e4fda142698243e1083a01d19e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "97e85e4e203f4848a880035f4714ca8e"
I20260808 05:02:56.318110 11466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [term 0 FOLLOWER]: Advancing to term 1
I20260808 05:02:56.318243 11160 leader_election.cc:302] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14415239f48b4717a8eaaee259707cff, ed3414e4fda142698243e1083a01d19e; no voters:
I20260808 05:02:56.318378 11564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 05:02:56.318526 11564 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [term 1 LEADER]: Becoming Leader. State: Replica: ed3414e4fda142698243e1083a01d19e, State: Running, Role: LEADER
I20260808 05:02:56.318603 11564 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } }
I20260808 05:02:56.318926 11466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed3414e4fda142698243e1083a01d19e in term 1.
I20260808 05:02:56.318966 11564 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed3414e4fda142698243e1083a01d19e. Latest consensus state: current_term: 1 leader_uuid: "ed3414e4fda142698243e1083a01d19e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.319024 11564 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:56.321254 11134 master_runner.cc:263] No leader master found from master ed3414e4fda142698243e1083a01d19e
I20260808 05:02:56.321333 11134 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 05:02:56.329468 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.318109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.3.174.189:33143)
0808 05:02:56.318155 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.318157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.321521 (+ 3364us) server_negotiation.cc:407] Connection header received
0808 05:02:56.321632 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.321634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.321645 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.321676 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.322441 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.323358 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.324502 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.324787 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.324829 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.326457 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.326459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.326459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.326467 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.328051 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.328490 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.328492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.328492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.328555 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.328773 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.328774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.328774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.328912 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.329007 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.329260 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.329374 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 05:02:56.329996 11610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 05:02:56.330212 11610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 05:02:56.338294 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.321564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41542 (local address 127.3.174.188:32897)
0808 05:02:56.321601 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.321603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.330352 (+ 8749us) server_negotiation.cc:407] Connection header received
0808 05:02:56.330448 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.330449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.330460 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.330490 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.331205 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.331888 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.332849 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.333479 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.333523 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.334882 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.334884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.334884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.334892 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.336977 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.337417 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.337418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.337419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.337497 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.337719 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.337720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.337720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.337847 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.337913 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.338108 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.338194 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 05:02:56.339159 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:56.320661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59026 (local address 127.3.174.190:43323)
0808 05:02:56.320750 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.320763 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.329462 (+ 8699us) server_negotiation.cc:407] Connection header received
0808 05:02:56.329849 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.329850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.329862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.329890 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.330729 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.331434 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.332465 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:56.332761 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:56.332800 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:56.335903 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 05:02:56.335922 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 05:02:56.335922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 05:02:56.335930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 05:02:56.337547 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.337952 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.337954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.337954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.338009 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 05:02:56.338196 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 05:02:56.338196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 05:02:56.338197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 05:02:56.338332 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 05:02:56.338407 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:56.338973 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 05:02:56.339062 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 05:02:56.342643 11466 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [term 1 FOLLOWER]: Refusing update from remote peer ed3414e4fda142698243e1083a01d19e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:56.343140 11598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:56.345002 11511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed3414e4fda142698243e1083a01d19e. Latest consensus state: current_term: 1 leader_uuid: "ed3414e4fda142698243e1083a01d19e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.345105 11511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:56.345940 11333 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [term 1 FOLLOWER]: Refusing update from remote peer ed3414e4fda142698243e1083a01d19e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 05:02:56.346290 11598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [LEADER]: Connected to new peer: Peer: permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 05:02:56.347569 11543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed3414e4fda142698243e1083a01d19e. Latest consensus state: current_term: 1 leader_uuid: "ed3414e4fda142698243e1083a01d19e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.347653 11543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:56.351490 11598 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ed3414e4fda142698243e1083a01d19e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.351615 11598 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:56.352133 11543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ed3414e4fda142698243e1083a01d19e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.352272 11543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:56.352486 11620 mvcc.cc:205] Tried to move back new op lower bound from 7316133381493145600 to 7316133381401522176. Current Snapshot: MvccSnapshot[applied={T|T < 7316133381493145600}]
I20260808 05:02:56.352600 11613 mvcc.cc:205] Tried to move back new op lower bound from 7316133381493145600 to 7316133381401522176. Current Snapshot: MvccSnapshot[applied={T|T < 7316133381493145600}]
I20260808 05:02:56.353685 11564 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ed3414e4fda142698243e1083a01d19e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.353791 11564 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e [sys.catalog]: This master's current role is: LEADER
I20260808 05:02:56.354301 11511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ed3414e4fda142698243e1083a01d19e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3414e4fda142698243e1083a01d19e" member_type: VOTER last_known_addr { host: "127.3.174.190" port: 43323 } } peers { permanent_uuid: "14415239f48b4717a8eaaee259707cff" member_type: VOTER last_known_addr { host: "127.3.174.189" port: 33143 } } peers { permanent_uuid: "97e85e4e203f4848a880035f4714ca8e" member_type: VOTER last_known_addr { host: "127.3.174.188" port: 32897 } } }
I20260808 05:02:56.354384 11511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 05:02:56.361866 11610 catalog_manager.cc:1347] Generated new cluster ID: c084ac91156d43b68430afe17f441feb
I20260808 05:02:56.361941 11610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 05:02:56.374368 11610 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 05:02:56.375106 11610 catalog_manager.cc:1504] Loading token signing keys...
I20260808 05:02:56.386130 11610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed3414e4fda142698243e1083a01d19e: Generated new TSK 0
I20260808 05:02:56.386505 11610 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 05:02:56.416841 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.415094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.3.174.189:33143)
0808 05:02:56.415140 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.415143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.416148 (+ 1005us) server_negotiation.cc:407] Connection header received
0808 05:02:56.416293 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.416294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.416306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.416332 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.416766 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46726: BlockingRecv error: recv got EOF from 127.0.0.1:46726 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 05:02:56.417729 11263 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.415564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59032 (local address 127.3.174.190:43323)
0808 05:02:56.415606 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.415608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.417081 (+ 1473us) server_negotiation.cc:407] Connection header received
0808 05:02:56.417175 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.417176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.417186 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.417322 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.417665 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59032: BlockingRecv error: recv got EOF from 127.0.0.1:59032 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 05:02:56.418514 11508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.415872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41550 (local address 127.3.174.188:32897)
0808 05:02:56.415928 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.415930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.417930 (+ 2000us) server_negotiation.cc:407] Connection header received
0808 05:02:56.418047 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.418048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.418070 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.418096 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.418453 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41550: BlockingRecv error: recv got EOF from 127.0.0.1:41550 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 05:02:56.420751 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.419899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46730 (local address 127.3.174.189:33143)
0808 05:02:56.419938 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.419939 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.420215 (+ 276us) server_negotiation.cc:407] Connection header received
0808 05:02:56.420323 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.420324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.420340 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.420368 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.420706 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46730: BlockingRecv error: recv got EOF from 127.0.0.1:46730 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 05:02:56.423751 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.422906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.3.174.189:33143)
0808 05:02:56.422939 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.422941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.423185 (+ 244us) server_negotiation.cc:407] Connection header received
0808 05:02:56.423282 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.423283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.423293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.423322 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.423702 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46746: BlockingRecv error: recv got EOF from 127.0.0.1:46746 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 05:02:56.427584 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.426795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46758 (local address 127.3.174.189:33143)
0808 05:02:56.426828 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.426830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.427066 (+ 236us) server_negotiation.cc:407] Connection header received
0808 05:02:56.427155 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.427156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.427165 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.427217 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.427536 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46758: BlockingRecv error: recv got EOF from 127.0.0.1:46758 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 05:02:56.433497 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.432638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.3.174.189:33143)
0808 05:02:56.432673 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.432675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.432948 (+ 273us) server_negotiation.cc:407] Connection header received
0808 05:02:56.433041 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.433042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.433052 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.433080 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.433451 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46764: BlockingRecv error: recv got EOF from 127.0.0.1:46764 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 05:02:56.443631 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.442806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46778 (local address 127.3.174.189:33143)
0808 05:02:56.442858 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.442860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.443080 (+ 220us) server_negotiation.cc:407] Connection header received
0808 05:02:56.443136 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.443137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.443150 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.443231 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.443571 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46778: BlockingRecv error: recv got EOF from 127.0.0.1:46778 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 05:02:56.461930 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.460985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.3.174.189:33143)
0808 05:02:56.461036 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.461038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.461170 (+ 132us) server_negotiation.cc:407] Connection header received
0808 05:02:56.461245 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.461247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.461261 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.461362 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.461862 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46790: BlockingRecv error: recv got EOF from 127.0.0.1:46790 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 05:02:56.495898 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.495137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46806 (local address 127.3.174.189:33143)
0808 05:02:56.495191 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.495193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.495270 (+ 77us) server_negotiation.cc:407] Connection header received
0808 05:02:56.495371 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.495372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.495384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.495440 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.495835 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46806: BlockingRecv error: recv got EOF from 127.0.0.1:46806 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 05:02:56.561873 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.561161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46820 (local address 127.3.174.189:33143)
0808 05:02:56.561226 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.561228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.561305 (+ 77us) server_negotiation.cc:407] Connection header received
0808 05:02:56.561398 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.561399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.561411 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.561471 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.561814 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46820: BlockingRecv error: recv got EOF from 127.0.0.1:46820 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 05:02:56.691716 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.691117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46824 (local address 127.3.174.189:33143)
0808 05:02:56.691171 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.691173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.691208 (+ 35us) server_negotiation.cc:407] Connection header received
0808 05:02:56.691285 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.691286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.691296 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.691343 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.691653 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46824: BlockingRecv error: recv got EOF from 127.0.0.1:46824 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 05:02:56.949695 11386 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 05:02:56.949026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46828 (local address 127.3.174.189:33143)
0808 05:02:56.949095 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:56.949096 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:56.949132 (+ 36us) server_negotiation.cc:407] Connection header received
0808 05:02:56.949209 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:56.949210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:56.949221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:56.949321 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 05:02:56.949631 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46828: BlockingRecv error: recv got EOF from 127.0.0.1:46828 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 05:02:57.243321 11540 catalog_manager.cc:1259] Loaded cluster ID: c084ac91156d43b68430afe17f441feb
I20260808 05:02:57.243404 11540 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e: loading cluster ID for follower catalog manager: success
I20260808 05:02:57.244175 11540 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e: acquiring CA information for follower catalog manager: success
I20260808 05:02:57.244457 11540 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 97e85e4e203f4848a880035f4714ca8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 05:02:57.306234 11561 catalog_manager.cc:1259] Loaded cluster ID: c084ac91156d43b68430afe17f441feb
I20260808 05:02:57.306332 11561 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff: loading cluster ID for follower catalog manager: success
I20260808 05:02:57.307206 11561 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff: acquiring CA information for follower catalog manager: success
I20260808 05:02:57.307524 11561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14415239f48b4717a8eaaee259707cff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 05:02:57.465528 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:57.462986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.3.174.189:33143)
0808 05:02:57.463089 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:57.463091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:57.463101 (+ 10us) server_negotiation.cc:407] Connection header received
0808 05:02:57.463150 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:57.463151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:57.463239 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:57.463322 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:57.463460 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.463966 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.464746 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.464948 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.465001 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:57.465125 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:57.465313 (+ 188us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:57.465390 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:57.465420 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 05:02:57.465440 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":57,"threads_started":1}
I20260808 05:02:57.469650 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:57.467340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41560 (local address 127.3.174.188:32897)
0808 05:02:57.467455 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:57.467457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:57.467465 (+ 8us) server_negotiation.cc:407] Connection header received
0808 05:02:57.467505 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:57.467506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:57.467544 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:57.467611 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:57.467696 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.468176 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.468854 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.469054 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.469116 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:57.469280 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:57.469412 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:57.469469 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:57.469546 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 05:02:57.469565 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260808 05:02:57.474198 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:57.471402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.3.174.189:33143)
0808 05:02:57.471440 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:57.471442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:57.471533 (+ 91us) server_negotiation.cc:407] Connection header received
0808 05:02:57.471580 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:57.471581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:57.471592 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:57.471637 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:57.471822 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.472436 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.473366 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.473662 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.473745 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:57.473925 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:57.473966 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:57.474012 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:57.474092 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 05:02:57.474118 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 05:02:57.474570 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:57.471653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.3.174.188:32897)
0808 05:02:57.471707 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:57.471708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:57.471856 (+ 148us) server_negotiation.cc:407] Connection header received
0808 05:02:57.471939 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:57.471940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:57.471952 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:57.472001 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:57.472187 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.472870 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.473827 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.474092 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.474150 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:57.474322 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:57.474354 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:57.474383 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:57.474478 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 05:02:57.474500 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 05:02:57.474714 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 05:02:57.471393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59046 (local address 127.3.174.190:43323)
0808 05:02:57.471492 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 05:02:57.471494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 05:02:57.471612 (+ 118us) server_negotiation.cc:407] Connection header received
0808 05:02:57.471702 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 05:02:57.471703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 05:02:57.471753 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 05:02:57.471828 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 05:02:57.471996 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.472740 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.473656 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 05:02:57.473957 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 05:02:57.474012 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 05:02:57.474409 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 05:02:57.474510 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 05:02:57.474566 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 05:02:57.474610 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 05:02:57.474636 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":47,"threads_started":1}
I20260808 05:02:57.476150 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 11134
I20260808 05:02:57.500028 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 11270
I20260808 05:02:57.511210 3770 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskihVxiI/build/release/bin/kudu with pid 11388
2026-08-08T05:02:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3129 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6323 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (281039 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST